-- Logs begin at Mon 2022-07-25 17:49:42 UTC, end at Mon 2022-07-25 19:05:13 UTC. -- Jul 25 18:41:51.860337 dv-cinder-825492-29-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 18:41:51.873008 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 18:41:51 2022] *** Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: nodename: dv-cinder-825492-29-iscsi Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: machine: x86_64 Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: clock source: unix Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: pcre jit disabled Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: detected number of CPU cores: 4 Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: current working directory: / Jul 25 18:41:51.875716 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: detected binary path: /usr/bin/uwsgi-core Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: your processes number limit is 47680 Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: your memory page size is 4096 bytes Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: detected max file descriptor number: 2048 Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: lock engine: pthread robust mutexes Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: thunder lock: enabled Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 18:41:51.876371 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 18:41:51.915566 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: Python main interpreter initialized at 0x56341b1369c0 Jul 25 18:41:51.915566 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: python threads support enabled Jul 25 18:41:51.915566 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: your server socket listen backlog is limited to 100 connections Jul 25 18:41:51.915566 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: your mercy for graceful operations on workers is 90 seconds Jul 25 18:41:51.915929 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 18:41:51.916005 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: *** Operational MODE: preforking *** Jul 25 18:41:51.916287 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: *** uWSGI is running in multiple interpreter mode *** Jul 25 18:41:51.916287 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: spawned uWSGI master process (pid: 86929) Jul 25 18:41:51.916497 dv-cinder-825492-29-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 18:41:51.919873 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: spawned uWSGI worker 1 (pid: 86941, cores: 1) Jul 25 18:41:51.919873 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: spawned uWSGI worker 2 (pid: 86942, cores: 1) Jul 25 18:41:51.919873 dv-cinder-825492-29-iscsi devstack@keystone.service[86929]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 18:41:57.587933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:57.587933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: from cryptography.utils import int_from_bytes Jul 25 18:41:57.588687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:57.588687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: from cryptography.utils import int_from_bytes Jul 25 18:41:57.589738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:57.589738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: from cryptography.utils import int_from_bytes Jul 25 18:41:57.590225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:57.590225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: from cryptography.utils import int_from_bytes Jul 25 18:41:57.639771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 18:41:57.639959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 18:41:57.640161 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] command line args: [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 18:41:57.640304 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 18:41:57.640402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 18:41:57.640487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 18:41:57.640569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ================================================================================ {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 18:41:57.640649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] command line args: [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 18:41:57.640835 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] admin_token = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.640929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 18:41:57.641091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ================================================================================ {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 18:41:57.641179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] config_dir = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641310 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] admin_token = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641500 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] config_source = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641592 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] config_dir = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] debug = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.641872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] config_source = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] debug = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642762 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.642985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643078 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_config_append = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_config_append = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.643945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_dir = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_dir = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_file = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644144 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_file = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644876 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.644998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645540 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.645909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.646073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.646203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.646452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.646586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.646716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.646840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647012 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.647954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] publish_errors = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648675 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] publish_errors = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.648922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649728 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649854 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.649980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.650116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] standard_threads = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.650265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] standard_threads = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.650398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.650517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.650641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.650873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651296 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] use_journal = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] use_journal = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651543 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] use_json = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] use_json = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] use_stderr = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.651945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] use_stderr = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.652073 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] use_syslog = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.652192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] use_syslog = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.652308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.652432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:57.652844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.652969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653338 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.653886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654326 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.654983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.655958 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.656115 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.656235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.656352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.656474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.656601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.656812 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.657052 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.657175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.657299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.657448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.657648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.657776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.658117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.658242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.663905 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664215 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664455 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.664940 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665584 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.665992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666571 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667011 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667011 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667220 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667340 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667459 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.667978 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.668087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.668234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.668879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.669924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.670990 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671116 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671610 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671737 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.671975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672455 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.672924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673063 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.673877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674248 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.674969 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.675850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676227 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676677 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.676926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677426 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677549 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.677912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678494 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.678998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679126 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.679996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.680895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.681035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.681410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.681529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.681765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.681899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.682973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683220 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683362 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.683842 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.684002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.684681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.684806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.684932 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.685951 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.686102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.686228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.686361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.686481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.686874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687163 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.external = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687808 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.external = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.687960 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.688084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.688239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.688529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.688667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.688804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.688927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.password = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689051 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.password = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] auth.token = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] auth.token = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689816 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.689946 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690256 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690505 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.690895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691386 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691651 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.691926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692083 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.692967 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693805 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.693947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.694086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.694288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.694429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.694550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.694658 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.698281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.698462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.698589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.698719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.698879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.699007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.699137 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.699285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.699405 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.699588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.700291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.700412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.700542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.700662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.700783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.700902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701144 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701273 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701393 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.701880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702506 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.702999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703120 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.703856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704497 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.704989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705228 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705347 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.705958 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706256 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706654 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.706920 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707484 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707726 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.707872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708144 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708787 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.708908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709758 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.709884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710787 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.710906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.711884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712615 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.712969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.713939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714455 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714827 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.714946 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.715064 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.715924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716535 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.716902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717021 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717386 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717875 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.717993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718393 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.718993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719350 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719586 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719586 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719849 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.719972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.720794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721626 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721726 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.721927 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.726953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.727955 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728564 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.728925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.729896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.730908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.731987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732114 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732410 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732647 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.732893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] role.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] role.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] role.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733254 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] role.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733501 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733880 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.733999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734405 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.734883 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735616 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735865 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.735984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.736951 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737814 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.737942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.738115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.738249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.738515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.738644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.738775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.738897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739016 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739500 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.739980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740832 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.740959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741199 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741554 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.741925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742676 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742816 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.742940 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743059 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743174 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743648 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.743966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.744322 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.744442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.744561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.744687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.744903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.745936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746367 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.connection = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.connection = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.746964 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.747917 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748156 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.748986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749224 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.749932 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750077 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.750803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.751029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.751147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.751266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:57.751385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 18:41:57.751514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 18:41:57.757152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.757389 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.757614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.757741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.757880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.758086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.766636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.767036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.801893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.802149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.802281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.802403 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.868946 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.869690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.923415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.923680 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.930712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:57.930950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021573 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021703 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021830 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.021953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.022148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.022300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.022422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.022543 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.022543 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.052471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.052693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.052828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.052956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.053095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.053307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.053486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.053701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.053833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.053938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.054063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.054271 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.054398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.054521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.065403 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.065710 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.078529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.078712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.078840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.078975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.079962 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.080083 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.080202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:58.097750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.097938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:58.105093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.105093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:58.105338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.105338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:58.105563 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.105696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.105818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:58.105939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.106601 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.106725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.106845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.106976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.107097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.107097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.110903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.111180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.111449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.111599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.111725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.111846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.112048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.112328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.120123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.123989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.124265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.124546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.124851 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.125130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.125383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.125650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.125919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.126209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.126477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.126740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.128577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.134816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.135305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.135799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.136310 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.136822 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.137347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.138441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.138948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.139443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.139947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.140448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.156317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.157993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.158294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.163592 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.163990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.164272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.170688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.171693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.171972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.172341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.172627 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.176555 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.176837 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.177116 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.177394 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.177668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.177939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.178079 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.178214 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.178393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.178520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.178679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.186845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.187125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.191659 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.191948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.192234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.192522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.192803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.193100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.193340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.193610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.200614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.200614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.210897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.210897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.210897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.213396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.213530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.213690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.217192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.218432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.220831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.221137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.221422 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.221701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.221973 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.222284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.223937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.224349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.229708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.233963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.237633 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.237933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.238248 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.238544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.238826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.239109 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.242082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.245147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.245147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.245147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.251496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.253236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.253577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.257037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.257319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.257599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.257883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.258179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.261934 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.262276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.262548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.262815 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.270834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.273798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.273798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.276150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.276433 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.276724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.277014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.277294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.278489 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.278771 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.279048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.279320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.279696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.279982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.280267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.291016 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.291299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.291583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.291866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.292244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.292545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.292827 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.299001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.302335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.305677 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.307665 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.309926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.310222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.310501 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.312408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.315714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.319751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.319947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.320087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.320935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.322630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.322776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.322927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.323064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.323198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.323384 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.323670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.330823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.332451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.332950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.333236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.333513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.335707 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.336056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.336419 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.336710 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.341469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.341744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.342599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.342599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.342599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.342996 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.343182 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.343457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.343722 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.343997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.344279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.344536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.344810 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.345067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.346045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.346345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.346639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.354595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:58.354886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.355168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.355445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.355726 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.356641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.356919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.357200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.357471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.357749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.358059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.358326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.358608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.358869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:58.384199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56341b1369c0 pid: 86941 (default app) Jul 25 18:41:58.387780 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4994f-fef4-4387-ab75-6e1bb0acfe20 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:58.388073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4994f-fef4-4387-ab75-6e1bb0acfe20 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:58.388348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4994f-fef4-4387-ab75-6e1bb0acfe20 None None] PATH_INFO: `/v3` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:58.392576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Mon Jul 25 18:41:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:58.398219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56341b1369c0 pid: 86942 (default app) Jul 25 18:42:07.273684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.274235 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.274322 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.275622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 2/2] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:07.283578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.283980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.284128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.353518 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: INFO dbcounter [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] Registered counter for database keystone Jul 25 18:42:07.375098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG oslo_db.sqlalchemy.engines [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 18:42:07.381653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writer thread running {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 18:42:07.556178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG passlib.handlers.bcrypt [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86942) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 18:42:07.556666 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG passlib.handlers.bcrypt [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86942) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 18:42:07.619449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:07.621118 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:07.748469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 1/3] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 25 18:42:07.893473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.962608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: INFO dbcounter [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] Registered counter for database keystone Jul 25 18:42:07.982738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG oslo_db.sqlalchemy.engines [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86941) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 18:42:07.989017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writer thread running {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 18:42:08.108350 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d49b17-9679-4fca-8fd0-986a5f58bc08 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:08.112626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:08.113294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.113570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.113840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.121979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:08.172376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WARNING keystone.common.rbac_enforcer.enforcer [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 18:42:08.172660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398610ff-3d3c-4e2a-adcc-8746d483f451 admin admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:08.182197 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 3/4] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Jul 25 18:42:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 18:42:08.188949 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:08.204196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc168a2-29aa-4b5b-a005-9b512db6f28e None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:08.208775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:08.210055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.210350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.210649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] PATH_INFO: `/v3/projects/16218857a48d4501b2b8db4e0e5b32f7` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.212164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] RBAC: Authorizing `identity:get_project(project_id=16218857a48d4501b2b8db4e0e5b32f7)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:08.272266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 18:42:08.272605 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82cca6-2ee4-4cd4-92aa-8f805042416b admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:08.277317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 2/5] 10.222.0.57 () {56 vars in 1207 bytes} [Mon Jul 25 18:42:08 2022] GET /identity/v3/projects/16218857a48d4501b2b8db4e0e5b32f7 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:10.290072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59834c8-f556-4395-975d-c6dc7f64ee30 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.290580 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59834c8-f556-4395-975d-c6dc7f64ee30 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.290692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59834c8-f556-4395-975d-c6dc7f64ee30 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.294737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 4/6] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:10.300251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.300695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.300783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.321605 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:10.323151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:10.331574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 3/7] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 18:42:10.353460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59834c8-f556-4395-975d-c6dc7f64ee30 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:10.365814 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59834c8-f556-4395-975d-c6dc7f64ee30 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:10.370198 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeacc65-2716-4ee9-ae3c-313534aa711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:10.370855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeacc65-2716-4ee9-ae3c-313534aa711e admin admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.371134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeacc65-2716-4ee9-ae3c-313534aa711e admin admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.371404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeacc65-2716-4ee9-ae3c-313534aa711e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.379952 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeacc65-2716-4ee9-ae3c-313534aa711e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:10.382345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeacc65-2716-4ee9-ae3c-313534aa711e admin admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:10.392598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 5/8] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Jul 25 18:42:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 18:42:10.398860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:10.408825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9f82-c7e9-412d-87bd-f2e1bdc10435 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:10.413337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e46c0-bc9b-45aa-8853-b01e1d94098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:10.414241 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e46c0-bc9b-45aa-8853-b01e1d94098b admin admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.414524 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e46c0-bc9b-45aa-8853-b01e1d94098b admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.414799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e46c0-bc9b-45aa-8853-b01e1d94098b admin admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.416212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e46c0-bc9b-45aa-8853-b01e1d94098b admin admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:10.418064 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e46c0-bc9b-45aa-8853-b01e1d94098b admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:10.419650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 4/9] 10.222.0.57 () {56 vars in 1198 bytes} [Mon Jul 25 18:42:10 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:12.523272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93047739-0c42-4c03-8e72-5aeba368f2fd None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.523788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93047739-0c42-4c03-8e72-5aeba368f2fd None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.523901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93047739-0c42-4c03-8e72-5aeba368f2fd None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.525179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 6/10] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:12.530765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.531183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.531395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.554097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:12.555308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:12.563648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31f931-d1db-4c7f-af72-c0d8ba0edf78 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.563931 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31f931-d1db-4c7f-af72-c0d8ba0edf78 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.564902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31f931-d1db-4c7f-af72-c0d8ba0edf78 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.565546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 7/11] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:12.570705 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed2250-d4db-4509-be85-76449340adda None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.570990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed2250-d4db-4509-be85-76449340adda None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.571434 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed2250-d4db-4509-be85-76449340adda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.616775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG passlib.handlers.bcrypt [None req-32ed2250-d4db-4509-be85-76449340adda None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86941) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 18:42:12.617202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG passlib.handlers.bcrypt [None req-32ed2250-d4db-4509-be85-76449340adda None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86941) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 18:42:12.639689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 5/12] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:12.661670 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.663093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-32ed2250-d4db-4509-be85-76449340adda None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:12.664684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-32ed2250-d4db-4509-be85-76449340adda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:12.686121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.766138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82168f-ac54-4e51-84b9-9c7910a891c4 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:12.767058 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 8/13] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:12.790452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed2250-d4db-4509-be85-76449340adda None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.800824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:12.801580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.801869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.802182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.884669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:12.886373 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:12.900210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed2250-d4db-4509-be85-76449340adda None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:12.902515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.910201 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.941312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:12.942406 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.942693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.942964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.954623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:12.966174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:12.967133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:12.967891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 9/14] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:12.973316 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.991734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 6/15] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:12.997486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.075426 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db589a5e-9b6d-43c0-b58a-9d1f1438d4e5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.097610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cf98a-fc08-45fd-8b76-2ad6be736b62 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.110299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.110978 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.111261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.111540 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.112305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:13.112606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.114073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.121643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:13.126914 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 10/16] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:13.133062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.136374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.136374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.136374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.136374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.147638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.149213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.161439 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:42:13.170761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:13.172606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:13.173692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:42:13.173692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:42:13.173692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:13.173692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:13.173692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 7/17] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:13.177956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.234632 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912689e1-9a51-45ff-a739-124ab852865a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.269948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.270740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.271025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.271300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.272863 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.274338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.275964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 11/18] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:13.278985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc2bb0-b26e-4888-94df-cfc1e1c32238 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.281310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.313782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.314489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.314772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.315049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.315782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:13.316090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.317643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.335842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 8/19] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:13.379496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:13.386738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbcf81-96ad-4598-83d9-6e94045ce4ff None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.424441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe57bed-7b77-49bd-a1b8-144601917964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.425293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57bed-7b77-49bd-a1b8-144601917964 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.425574 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57bed-7b77-49bd-a1b8-144601917964 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.425872 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57bed-7b77-49bd-a1b8-144601917964 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.427150 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe57bed-7b77-49bd-a1b8-144601917964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.428629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe57bed-7b77-49bd-a1b8-144601917964 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.430321 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 12/20] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:13.437195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.475122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:13.547403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54bc4a-6bf4-42e2-bc75-75ca03e091e8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.586397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.587409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.587865 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.588311 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.589353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.domain.id=default, user.id=2ba410fb97cd412fa30f98b46a43dd28) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:13.589660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.591197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315b6d-d352-4b5a-9143-63ef413f7226 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.599394 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 9/21] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/role_assignments?user.id=2ba410fb97cd412fa30f98b46a43dd28&scope.domain.id=default&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.696574 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699d481-9a6d-4131-b5c9-b97e12728a38 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.697638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699d481-9a6d-4131-b5c9-b97e12728a38 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.698264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699d481-9a6d-4131-b5c9-b97e12728a38 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.700210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 13/22] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:15.705484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.706450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.706984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.731482 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:15.733015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:15.820849 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 10/23] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:15.854048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699d481-9a6d-4131-b5c9-b97e12728a38 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.975352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699d481-9a6d-4131-b5c9-b97e12728a38 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.012769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.013914 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.014426 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.014955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.035001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.036901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d4275deb-46af-4ed2-80a6-b90cc6bb4ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:42:16.056899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:16.057750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:16.059032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:42:16.059032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:42:16.059032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:16.059032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:16.059032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 14/24] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:16.064820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.114695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.123365 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697411e-9dd8-4d4e-bb80-cef82c030296 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.123657 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697411e-9dd8-4d4e-bb80-cef82c030296 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.123962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697411e-9dd8-4d4e-bb80-cef82c030296 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.126036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 15/25] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:16.131345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f042d3-d74c-4e98-8d25-646d6019f640 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.131634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f042d3-d74c-4e98-8d25-646d6019f640 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.131911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f042d3-d74c-4e98-8d25-646d6019f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.167125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-93f042d3-d74c-4e98-8d25-646d6019f640 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:16.168366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-93f042d3-d74c-4e98-8d25-646d6019f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:16.198531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.237709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452b9a3-f769-4f52-9d2e-b1df6888fac1 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.279994 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 16/26] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:16.281865 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4851c47-ea77-4b43-9084-6f0f7a707739 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.282350 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4851c47-ea77-4b43-9084-6f0f7a707739 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.282773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4851c47-ea77-4b43-9084-6f0f7a707739 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.288714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 17/27] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:16.290595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2266d-e934-4f77-8ae4-9eff411c3235 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.291067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2266d-e934-4f77-8ae4-9eff411c3235 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.291494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2266d-e934-4f77-8ae4-9eff411c3235 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.293989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 18/28] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:16.295541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b638057-5712-4fb2-85dd-83178c5d3a1c None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.296006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b638057-5712-4fb2-85dd-83178c5d3a1c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.296453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b638057-5712-4fb2-85dd-83178c5d3a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.329600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-1b638057-5712-4fb2-85dd-83178c5d3a1c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:16.332127 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1b638057-5712-4fb2-85dd-83178c5d3a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:16.401600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.403094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.403549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.404266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.406461 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:16.407195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.409633 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.422958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 19/29] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:16.422958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.422958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.422958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.432403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 11/30] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:16.439878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.446031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:16.447126 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:16.569129 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 20/31] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:16.571088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.576149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.584514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.609661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e405ff-a229-446e-92c8-53d91ed7a143 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.646755 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.647810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.648306 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.648780 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.649729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.651354 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.703314 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41ba71-c9df-4688-905e-cee00b70a574 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.739620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.740345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.740630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.741024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.741741 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.743241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.873305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 12/32] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 25 18:42:16 2022] POST /identity/v3/roles => generated 225 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:16.875697 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.000131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebf7ef-8786-44f4-b1b5-0581aee66334 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.038930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.040134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.040615 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.041083 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.054429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:17.064747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:17.065625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:17.066412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:17.069238 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 13/33] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:17.070326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.084353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.126521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 21/34] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:42:16 2022] POST /identity/v3/projects => generated 301 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:17.128590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.212686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.235327 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9db7-df5d-4b44-89e1-1350363fd1a1 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.267992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8333b1-8c75-4f4a-941e-1d721fe796e5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.275886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562a62-49cd-4494-b6af-a31a6809b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.277719 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562a62-49cd-4494-b6af-a31a6809b2cb None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.278190 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562a62-49cd-4494-b6af-a31a6809b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.278605 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562a62-49cd-4494-b6af-a31a6809b2cb None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.279582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36562a62-49cd-4494-b6af-a31a6809b2cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.282058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36562a62-49cd-4494-b6af-a31a6809b2cb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.308830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.309856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.310231 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.310425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.311229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:17.311536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.313840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a054a-79f9-4b1d-b2ff-fd5f2f97a1fd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.324512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 22/35] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.326352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fab36-beab-4124-aa0b-bdd1ce36b113 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.330903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fab36-beab-4124-aa0b-bdd1ce36b113 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.330903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fab36-beab-4124-aa0b-bdd1ce36b113 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.330903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 23/36] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:17.332338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fab36-beab-4124-aa0b-bdd1ce36b113 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.363766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 14/37] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 25 18:42:17 2022] POST /identity/v3/roles => generated 219 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:17.366453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ecbe5-6e36-42b0-835e-91cfdb73c127 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.366745 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ecbe5-6e36-42b0-835e-91cfdb73c127 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.367400 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ecbe5-6e36-42b0-835e-91cfdb73c127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.402335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-c49ecbe5-6e36-42b0-835e-91cfdb73c127 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:17.403913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c49ecbe5-6e36-42b0-835e-91cfdb73c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:17.470208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fab36-beab-4124-aa0b-bdd1ce36b113 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.483441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.486350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 15/38] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:17.495652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8286d0-af7e-4f35-a71e-001de868db39 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.495944 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8286d0-af7e-4f35-a71e-001de868db39 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.496225 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8286d0-af7e-4f35-a71e-001de868db39 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.497787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 16/39] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:17.507785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.508064 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.508380 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.509061 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.513745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.514689 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.515521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.515806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.519655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.521070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.523928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 24/40] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.525325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.536208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:17.538363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:17.617447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 17/41] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:17.619898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.656497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd61a7a-7831-4ae8-9540-223559fec2bb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.689885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.690609 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.690894 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.691165 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.691909 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.693291 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.724823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833a21d-25a8-45ef-aae0-ecaea5c5d5c1 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.751375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.758948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.759739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.760021 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.760315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.761466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.762945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.764544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 18/42] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.766100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.780512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 25/43] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 25 18:42:17 2022] POST /identity/v3/roles => generated 223 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:17.780512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.833297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.883067 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b285c3d-ff63-4235-a230-852745faa36c None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.899633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61e9cf-5ddb-4798-9b45-252b134ea762 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.930056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.930887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.931175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.931458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.932701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.934358 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.936316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 19/44] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.941997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.948242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ec0b8-df1d-4b40-a8e3-c4de9f7dcd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.949211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ec0b8-df1d-4b40-a8e3-c4de9f7dcd48 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.949566 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ec0b8-df1d-4b40-a8e3-c4de9f7dcd48 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.949848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ec0b8-df1d-4b40-a8e3-c4de9f7dcd48 None admin] PATH_INFO: `/v3/domains/default/users/2ba410fb97cd412fa30f98b46a43dd28/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.952337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ec0b8-df1d-4b40-a8e3-c4de9f7dcd48 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2ba410fb97cd412fa30f98b46a43dd28, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.953803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ec0b8-df1d-4b40-a8e3-c4de9f7dcd48 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:18.007819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:18.044503 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231a7e-e3a4-4ffe-931f-696cab33ce2f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:18.076801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6717a-04b5-4a22-9f17-082d03686402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:18.077483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6717a-04b5-4a22-9f17-082d03686402 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:18.077769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6717a-04b5-4a22-9f17-082d03686402 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:18.078070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6717a-04b5-4a22-9f17-082d03686402 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:18.078779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6717a-04b5-4a22-9f17-082d03686402 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:18.080235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6717a-04b5-4a22-9f17-082d03686402 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:18.102208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 26/45] 10.222.0.57 () {58 vars in 1361 bytes} [Mon Jul 25 18:42:17 2022] PUT /identity/v3/domains/default/users/2ba410fb97cd412fa30f98b46a43dd28/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:18.224862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 20/46] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:42:17 2022] POST /identity/v3/projects => generated 312 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:20.106306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1763088-731b-4cfe-9673-187b477c5539 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.106845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1763088-731b-4cfe-9673-187b477c5539 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.106931 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1763088-731b-4cfe-9673-187b477c5539 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.108247 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 27/47] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:20.113606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d44ce-3951-48ac-81f1-699d1c4d4a2a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.114045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d44ce-3951-48ac-81f1-699d1c4d4a2a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.114192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d44ce-3951-48ac-81f1-699d1c4d4a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.132345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-f21d44ce-3951-48ac-81f1-699d1c4d4a2a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:20.133521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-f21d44ce-3951-48ac-81f1-699d1c4d4a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:20.198937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 21/48] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:20.224780 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1763088-731b-4cfe-9673-187b477c5539 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.254453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb0334-b8f9-4022-ac95-b15040ae6c23 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.254749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb0334-b8f9-4022-ac95-b15040ae6c23 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.255036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb0334-b8f9-4022-ac95-b15040ae6c23 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.256466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 22/49] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:20.261467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.261760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.262063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.279519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:20.280721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:20.324010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.328680 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1763088-731b-4cfe-9673-187b477c5539 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.335746 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.348559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 23/50] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:20.363999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.364733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.365022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.365301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.376205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.376347 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:20.385155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:20.386004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:20.386823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 28/51] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:20.393475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.479433 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9cbbf-8d0a-4078-af7d-c6ce97e5c98c None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.494564 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbc1d3-1993-4a53-95d5-1e989ea01efb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.515295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.516101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.516389 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.516669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.517871 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.519430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.521061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 24/52] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.526568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.528286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.528965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.529252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.529532 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.530335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:20.530643 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.532033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.542074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 29/53] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:20.545917 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.575956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.579769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.633475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1949ab-36b4-4ff5-ac1a-1468ca03af96 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.648385 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953adec-9839-4ce2-8e4d-5325b61d0261 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.666909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953b378-b363-4233-9a32-5c22946e4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.667570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953b378-b363-4233-9a32-5c22946e4e11 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.667856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953b378-b363-4233-9a32-5c22946e4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.668139 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953b378-b363-4233-9a32-5c22946e4e11 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.668844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953b378-b363-4233-9a32-5c22946e4e11 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.670338 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953b378-b363-4233-9a32-5c22946e4e11 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.682598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.683480 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.683775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.684056 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.685530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.687016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.689232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 30/54] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:20.694745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.787342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.792976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765268-db5e-4c19-9465-eaf8bb7539ea None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.826721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.827513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.827824 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.828100 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.829286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.830768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.832458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 31/55] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:20.837998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.864684 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 25/56] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Jul 25 18:42:20 2022] POST /identity/v3/projects => generated 298 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:20.945440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e221dc-95e7-44dc-bee9-4ddd399cb8c2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.981815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99754-0072-416d-8cee-502d322f710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.982517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99754-0072-416d-8cee-502d322f710e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.982811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99754-0072-416d-8cee-502d322f710e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.983083 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99754-0072-416d-8cee-502d322f710e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.983933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99754-0072-416d-8cee-502d322f710e None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.domain.id=default, user.id=2ba410fb97cd412fa30f98b46a43dd28) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:20.984310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99754-0072-416d-8cee-502d322f710e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.985777 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99754-0072-416d-8cee-502d322f710e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.994809 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 32/57] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/role_assignments?user.id=2ba410fb97cd412fa30f98b46a43dd28&scope.domain.id=default&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:22.916639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbdf5d-0bd6-4e00-b5a8-47afaec7c524 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.917141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbdf5d-0bd6-4e00-b5a8-47afaec7c524 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.917141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbdf5d-0bd6-4e00-b5a8-47afaec7c524 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.918590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 26/58] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:22.924336 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.924748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.924906 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.947506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:22.947506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:22.969502 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.013219 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 33/59] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:23.037727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbdf5d-0bd6-4e00-b5a8-47afaec7c524 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.131292 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.137586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbdf5d-0bd6-4e00-b5a8-47afaec7c524 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.170283 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01c056-87fd-4cac-b030-f07a68202d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.171012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c056-87fd-4cac-b030-f07a68202d31 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.171296 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c056-87fd-4cac-b030-f07a68202d31 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.171573 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c056-87fd-4cac-b030-f07a68202d31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.172810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c056-87fd-4cac-b030-f07a68202d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.174297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c056-87fd-4cac-b030-f07a68202d31 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.175897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 27/60] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:23.181696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.276800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed4d7d-dc2f-4326-b87d-51d102625d19 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.309073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867118f-be65-4e76-b68b-e5880d1a83ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.309734 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867118f-be65-4e76-b68b-e5880d1a83ef None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.310037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867118f-be65-4e76-b68b-e5880d1a83ef None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.310318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867118f-be65-4e76-b68b-e5880d1a83ef None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.311284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867118f-be65-4e76-b68b-e5880d1a83ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.313229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867118f-be65-4e76-b68b-e5880d1a83ef None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.400560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 34/61] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:23 2022] POST /identity/v3/users => generated 275 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:26.592106 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f765d-48ba-4306-8d8d-91bf62284d9b None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.593092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f765d-48ba-4306-8d8d-91bf62284d9b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.593577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f765d-48ba-4306-8d8d-91bf62284d9b None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.595820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 28/62] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:26.601478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.602391 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.602878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.630038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:26.630038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:26.644438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:26.702367 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c211e6-e6a7-4a73-b71c-707564ef3174 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.702703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c211e6-e6a7-4a73-b71c-707564ef3174 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.702936 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c211e6-e6a7-4a73-b71c-707564ef3174 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.705108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 29/63] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:26.719598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 35/64] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:26.721968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fce51-3c73-4415-a15c-fd9d757c868b None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.722553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fce51-3c73-4415-a15c-fd9d757c868b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.723050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fce51-3c73-4415-a15c-fd9d757c868b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.757221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-fc2fce51-3c73-4415-a15c-fd9d757c868b None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:26.757428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:26.758498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-fc2fce51-3c73-4415-a15c-fd9d757c868b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:26.874247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 30/65] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:26.877108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392ec76-2ceb-4c38-a863-a6539f404663 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.877629 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392ec76-2ceb-4c38-a863-a6539f404663 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.878140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392ec76-2ceb-4c38-a863-a6539f404663 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.883614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 31/66] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:26.896827 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571a7f7-d6ef-455f-bafd-f75f9359fd42 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.897307 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571a7f7-d6ef-455f-bafd-f75f9359fd42 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.897789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571a7f7-d6ef-455f-bafd-f75f9359fd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.904683 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e499c8-5b97-4dff-bf39-a4f54f643d40 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:26.927021 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-f571a7f7-d6ef-455f-bafd-f75f9359fd42 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:26.929185 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-f571a7f7-d6ef-455f-bafd-f75f9359fd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:26.943893 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:26.944958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.945403 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.945830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.958070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:26.965868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:26.966795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:26.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 36/67] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:26.968249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:26.972033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.024899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 32/68] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:27.026612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07ebd-9123-43d8-991a-d7b323edcec4 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.027119 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07ebd-9123-43d8-991a-d7b323edcec4 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.027683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07ebd-9123-43d8-991a-d7b323edcec4 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.029170 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 33/69] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:27.034196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f07ebd-9123-43d8-991a-d7b323edcec4 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.064614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.078929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ba0-7f71-44ba-857a-0da0c4488a6f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.114763 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcfb24-2dc3-435d-8df5-6b1048dc5a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.115675 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcfb24-2dc3-435d-8df5-6b1048dc5a16 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.115963 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcfb24-2dc3-435d-8df5-6b1048dc5a16 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.116243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcfb24-2dc3-435d-8df5-6b1048dc5a16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.129570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcfb24-2dc3-435d-8df5-6b1048dc5a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-33fcfb24-2dc3-435d-8df5-6b1048dc5a16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:27.138101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:27.138972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:27.139730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 37/70] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:27.140990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cbc45-39df-48f5-bf25-487896037210 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.140990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cbc45-39df-48f5-bf25-487896037210 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.141240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cbc45-39df-48f5-bf25-487896037210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.157824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f07ebd-9123-43d8-991a-d7b323edcec4 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.161887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-274cbc45-39df-48f5-bf25-487896037210 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:27.163117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-274cbc45-39df-48f5-bf25-487896037210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:27.201323 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.202032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.202359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.202685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.203486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.203763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.205182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.217819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 34/71] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.218693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.241633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.252962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 38/72] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:27.254267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cbc45-39df-48f5-bf25-487896037210 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.325807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.340712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ceffc-8484-42f3-9a63-8554ce9b93f5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.362086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cbc45-39df-48f5-bf25-487896037210 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.374619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.375425 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.375720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.376003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.386330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:27.394677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:27.395579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:27.396315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 35/73] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:27.396932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.398075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.398754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.399042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.399321 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.400087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.400387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.401876 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.415203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 39/74] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.415203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.468622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.499299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5606b6d-256c-42f1-b25b-b883c3c133e7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.513032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abe5d-22fd-43d9-8572-309481183fc1 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.535908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bfdbc-1c03-463a-99f7-0b8b1435a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.536796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bfdbc-1c03-463a-99f7-0b8b1435a77b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.537085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bfdbc-1c03-463a-99f7-0b8b1435a77b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.537371 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bfdbc-1c03-463a-99f7-0b8b1435a77b None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.549004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.549807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.550132 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.550404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.552243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bfdbc-1c03-463a-99f7-0b8b1435a77b None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.553759 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bfdbc-1c03-463a-99f7-0b8b1435a77b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.555513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 36/75] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.556856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ea697-0c95-4df2-9218-df0a1baad008 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.557255 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ea697-0c95-4df2-9218-df0a1baad008 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.557426 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ea697-0c95-4df2-9218-df0a1baad008 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.558863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 37/76] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:27.559470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ea697-0c95-4df2-9218-df0a1baad008 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.563698 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.566163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:27.574377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:27.583584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 40/77] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:27.584523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.665014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ea697-0c95-4df2-9218-df0a1baad008 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.683915 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d632d88-dbd3-4134-9a79-0bfaa7608720 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.700268 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.700418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.700952 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.701345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.701519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.702293 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.702600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.703996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.712690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 38/78] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.713461 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.719478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21055f7b-685e-4283-aced-5cf455869fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.720896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055f7b-685e-4283-aced-5cf455869fc7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.721237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055f7b-685e-4283-aced-5cf455869fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.721565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055f7b-685e-4283-aced-5cf455869fc7 None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.722829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055f7b-685e-4283-aced-5cf455869fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.724709 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055f7b-685e-4283-aced-5cf455869fc7 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.727152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 41/79] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.728618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.728912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.729339 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.750815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:27.752037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:27.780377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.815242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094aa6db-eff1-43c3-88f5-aa776326e792 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.818094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 42/80] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:27.818943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.851931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.852588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.852867 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.853141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.919732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.921104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.924883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e50574-7a9a-4a2b-a9ab-47433813481c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.939130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.959743 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.960415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.960700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.960979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.961731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.962082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.963495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.972978 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 43/81] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.974155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.984853 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.019934 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 39/82] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:28.020793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.073221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b78c40-1089-4a7c-bf5c-665f9d08c8ce None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.106897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.107781 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.108072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.108350 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.109567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.111056 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.112709 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 44/83] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:28.113470 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.124803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a270f-87ea-4918-bb49-5ddb3a552df2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.159356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.160034 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.160321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.160444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.160600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.194499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.214345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96d1b-d4f5-4bc8-92c2-02c836df186b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.228945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.230450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.248461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.249261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.249569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.249853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.251069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.253047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.254723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 45/84] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:28.255487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.328415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 40/85] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:28.329294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.359146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39209d7c-0182-4cca-876a-1b2fbb62f809 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.372108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.393775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.394788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.395078 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.395356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.395884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.396562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.398104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.399765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 46/86] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:28.400640 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.429797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7683fc-b64c-4e2b-811b-137581978b20 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.471619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.472480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.472761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.473034 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.474324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.480435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.480435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 41/87] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:28.480435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.515597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb22467-7de2-4730-aa5b-a04141123aaf None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.559944 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.560620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.560904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.561184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.583409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6276d-37b5-4dbd-92d7-1aa4d56cbebc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.611694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.619672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.620350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.620630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.620907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.621618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.623173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.632974 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.634535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.634742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.732469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 47/88] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:28.733314 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.744401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 42/89] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:28 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:28.745282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.828805 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.848895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5a4da-30ed-4374-a648-64ee9e286ce6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.862242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c209df-a21a-4e92-ac62-7c20b6b61587 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.886856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.887757 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.888024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.888300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.889521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.891006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.893101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 48/90] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:28.893511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.900470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.901140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.901429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.901711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.976521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.999881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.999881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.020955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255defe-e162-4064-b537-63249f3c1db3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.079100 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.080460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.080899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.081177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.083561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=9134c5ce0660415e9a3bc4b4ecb08e82) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:29.083561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.084047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.090829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 49/91] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/role_assignments?user.id=9134c5ce0660415e9a3bc4b4ecb08e82&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:29.091567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.109295 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.132915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 43/92] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:29.133694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.258264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.269384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65ff8-33ce-4b9e-9973-db737eaab4c4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.295598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4ed8d-fc76-4a1a-a9b3-03c684839607 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.317570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.318553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.318986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.319423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.320976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] RBAC: Adding query filter params (role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=9134c5ce0660415e9a3bc4b4ecb08e82) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:29.321449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.323678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.331681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 44/93] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/role_assignments?user.id=9134c5ce0660415e9a3bc4b4ecb08e82&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:29.332945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.359455 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.360472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.360785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.361069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.362396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.363934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.365713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 50/94] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:29.374046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.485549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb0ce-7a35-40e4-8249-16c07e78dfbc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.530487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.554094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134467-9193-4b36-a0f7-c27213d492ff None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.567252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.568150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.576222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.576222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] PATH_INFO: `/v3/projects/77837403f948450491b7b0fcaf459861` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.576222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] RBAC: Authorizing `identity:get_project(project_id=77837403f948450491b7b0fcaf459861)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.576222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.577352 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 45/95] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/projects/77837403f948450491b7b0fcaf459861 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:29.588272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.602682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.603383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.603675 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.604081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.604954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=2ba410fb97cd412fa30f98b46a43dd28) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:29.605299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.606760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972be8dd-f769-469e-a12d-01377b949755 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.613488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 51/96] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/role_assignments?user.id=2ba410fb97cd412fa30f98b46a43dd28&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:29.646183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.785697 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df35d17-ac9c-4402-a1ea-26ebbed77612 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.838773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.839690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.840184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.840634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.841667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=77837403f948450491b7b0fcaf459861, user.id=9134c5ce0660415e9a3bc4b4ecb08e82) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:29.842185 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.843811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0efe0-a4a7-4b69-9b2b-5523783797eb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.854896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 46/97] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/role_assignments?user.id=9134c5ce0660415e9a3bc4b4ecb08e82&scope.project.id=77837403f948450491b7b0fcaf459861&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:31.847664 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d57da-4da5-42bb-99c1-d618475698f4 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.848137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d57da-4da5-42bb-99c1-d618475698f4 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.848137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d57da-4da5-42bb-99c1-d618475698f4 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.849777 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 52/98] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:31.854837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.855261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.855406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.883027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:31.884475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:31.902628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd58a-c8a4-4787-b4a4-eabc5ac4174e None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.903325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd58a-c8a4-4787-b4a4-eabc5ac4174e None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.903802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd58a-c8a4-4787-b4a4-eabc5ac4174e None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.905471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 53/99] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:31.911587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.912134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.912620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.933724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:31.935354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:31.963303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 47/100] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:31.989478 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.012800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.032355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 54/101] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:32.089097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.106360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec851e5d-14e8-4f39-abd1-16c925c2a067 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.141781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e235a-3578-43e0-8b8c-3127ea23d367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.142685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e235a-3578-43e0-8b8c-3127ea23d367 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.143501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e235a-3578-43e0-8b8c-3127ea23d367 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.143781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e235a-3578-43e0-8b8c-3127ea23d367 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.155337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e235a-3578-43e0-8b8c-3127ea23d367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-271e235a-3578-43e0-8b8c-3127ea23d367 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:32.163366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:32.164562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:32.165599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:32.167691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 48/102] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:32.169164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54389d71-e883-497e-a4c8-5032192e770e None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.169610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54389d71-e883-497e-a4c8-5032192e770e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.170054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54389d71-e883-497e-a4c8-5032192e770e None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.171485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 49/103] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:32.178057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.178218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.178475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.206642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.219832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:32.221075 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:32.246879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3f1f2-9ac7-4683-a51b-7fd3c28c9b1a None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.315481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6be50-79f1-448e-a7b8-b43244168b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.316289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6be50-79f1-448e-a7b8-b43244168b12 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.316639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6be50-79f1-448e-a7b8-b43244168b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.316924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6be50-79f1-448e-a7b8-b43244168b12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.335039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6be50-79f1-448e-a7b8-b43244168b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-73e6be50-79f1-448e-a7b8-b43244168b12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:32.344006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:32.345018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:32.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:32.347456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 50/104] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:32.348145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.348984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 55/105] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:32.350461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50be2f-8a28-47b0-8efe-b0008f357d95 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.350928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50be2f-8a28-47b0-8efe-b0008f357d95 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.351379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50be2f-8a28-47b0-8efe-b0008f357d95 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.352815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 56/106] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:32.354555 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50be2f-8a28-47b0-8efe-b0008f357d95 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.368149 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.463737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50be2f-8a28-47b0-8efe-b0008f357d95 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.493794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e32c6-9760-4428-a0c8-b272e5b20ba0 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.514036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.514036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.514036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.514036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.514472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:32.514539 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.516396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01802b5-0f04-4307-a132-3c01281a3094 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.517972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.528729 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 57/107] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:32.530772 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.531138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.531491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.541251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1881b-2999-4a55-b197-348822202625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.541997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1881b-2999-4a55-b197-348822202625 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.542551 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1881b-2999-4a55-b197-348822202625 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.542829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1881b-2999-4a55-b197-348822202625 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.543602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1881b-2999-4a55-b197-348822202625 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:32.543910 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1881b-2999-4a55-b197-348822202625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.545335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1881b-2999-4a55-b197-348822202625 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.557854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:32.558270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 51/108] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:32.559647 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1881b-2999-4a55-b197-348822202625 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.560505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:32.652903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 58/109] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:32.654349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.654815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.735887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1881b-2999-4a55-b197-348822202625 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.773411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.774578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.775143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.775606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.790955 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:32.803056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:32.804141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:32.805084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:32.807895 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 52/110] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:32.808832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.824480 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec7a2-367c-4b75-a481-5514c896d870 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.853001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.862650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.863722 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.864144 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.864553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] PATH_INFO: `/v3/users/0496d401474c4cf9af323e0fd0d2910a` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.883324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] RBAC: Authorizing `identity:get_user(user_id=0496d401474c4cf9af323e0fd0d2910a)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.885423 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.887947 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 59/111] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/users/0496d401474c4cf9af323e0fd0d2910a => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:32.888993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.927788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca233-6fc5-45f4-8791-13900b98ed15 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.955968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.963746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.968641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.968641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.968641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.968641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.968641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.969630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 53/112] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:32.970253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224e18-26d3-4967-9422-774edc976a12 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.042566 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7ca78-e04e-48c2-8b69-a366aa978f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.043372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7ca78-e04e-48c2-8b69-a366aa978f65 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.043657 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7ca78-e04e-48c2-8b69-a366aa978f65 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.043930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7ca78-e04e-48c2-8b69-a366aa978f65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.045088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7ca78-e04e-48c2-8b69-a366aa978f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.046581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7ca78-e04e-48c2-8b69-a366aa978f65 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.048389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 60/113] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.049990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375738e-a1e1-4a49-96ea-d0f192590274 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.050299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375738e-a1e1-4a49-96ea-d0f192590274 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.050577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375738e-a1e1-4a49-96ea-d0f192590274 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.054069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 61/114] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:33.054069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375738e-a1e1-4a49-96ea-d0f192590274 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.076918 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69098b33-11e2-4659-931d-169dc6935dd8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.111391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.112094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.112384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.112668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.112904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.113428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:33.113740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.115318 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.123574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 54/115] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.124361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.161194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375738e-a1e1-4a49-96ea-d0f192590274 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.183725 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.195886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022da2ac-25fc-4a64-93a9-67573319d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.196549 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022da2ac-25fc-4a64-93a9-67573319d035 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.197776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022da2ac-25fc-4a64-93a9-67573319d035 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.197776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022da2ac-25fc-4a64-93a9-67573319d035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.227436 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd3667-0790-4d1b-8981-421cdfa764a4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.262509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369f63e-0180-4ca0-bc36-4cb0c795ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.263381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369f63e-0180-4ca0-bc36-4cb0c795ad92 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.263671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369f63e-0180-4ca0-bc36-4cb0c795ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.263955 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369f63e-0180-4ca0-bc36-4cb0c795ad92 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.265174 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369f63e-0180-4ca0-bc36-4cb0c795ad92 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.266672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369f63e-0180-4ca0-bc36-4cb0c795ad92 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.266939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022da2ac-25fc-4a64-93a9-67573319d035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.268922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 55/116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.269024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022da2ac-25fc-4a64-93a9-67573319d035 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.270161 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145d5d-f566-40fd-8fba-a9c1700e8587 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.270455 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145d5d-f566-40fd-8fba-a9c1700e8587 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.270734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145d5d-f566-40fd-8fba-a9c1700e8587 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.272691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 56/117] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:33.272691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145d5d-f566-40fd-8fba-a9c1700e8587 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.323704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.378551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 62/118] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:33.378771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145d5d-f566-40fd-8fba-a9c1700e8587 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.379994 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.380221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.380508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.398764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:33.399992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:33.408103 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.417842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.418137 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.418387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.418663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.419368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.420830 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.470092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 63/119] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:33.471008 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.509774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 57/120] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:42:33 2022] POST /identity/v3/projects => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:33.510642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.575422 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.596016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220c19f-e29c-473c-ab90-8baa7a8ac8e3 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.637093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade317a-7a28-43f2-8c78-b612716c7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.638003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade317a-7a28-43f2-8c78-b612716c7e28 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.638312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade317a-7a28-43f2-8c78-b612716c7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.638585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade317a-7a28-43f2-8c78-b612716c7e28 None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.639915 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade317a-7a28-43f2-8c78-b612716c7e28 None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.641400 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade317a-7a28-43f2-8c78-b612716c7e28 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.643736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 64/121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.644974 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.645260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.645535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.654646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e23bf-ebf6-4f3b-a35c-ee92ff58048d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.666245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:33.667468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:33.701253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.702214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.702501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.702777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/9134c5ce0660415e9a3bc4b4ecb08e82/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.704842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb86611f7b4363a6d822a20268eb19, user_id=9134c5ce0660415e9a3bc4b4ecb08e82, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.708019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.739403 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 65/122] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:33.745620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.783830 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 58/123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:33 2022] PUT /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/9134c5ce0660415e9a3bc4b4ecb08e82/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:33.784729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.848272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.856578 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.888245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661bf70-7931-42e7-88d4-4230b07de9ff None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.929842 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.931238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.931781 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.932258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.933806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.935528 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.937933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 66/124] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.939036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.939492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348dd3-85aa-4aaf-b168-43fc35ed4195 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.985272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.986127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.986460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.987282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.999661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:34.008391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:34.009253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:34.010036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 59/125] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:34.010682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.057623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8557f-febe-4e64-8cb3-4c8fbd9341a9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.092179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.093055 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.093341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.093619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.094847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.096348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.098378 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 67/126] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.099172 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.130448 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.137127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.143458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc91ce3-68ee-435a-8a22-d4e6f51a8778 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.186642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.187703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.187990 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.188267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.206396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:34.217439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:34.218608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:34.219519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:34.220088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61b1da-1696-40d5-8473-fe7457191377 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.223796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 60/127] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:34.223796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.273165 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.273860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.274178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.274520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.275942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] RBAC: Adding query filter params (role.id=c855898533554ff5b216819dc74c80e8, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=0496d401474c4cf9af323e0fd0d2910a) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:34.276260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.278226 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.290112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 68/128] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/role_assignments?user.id=0496d401474c4cf9af323e0fd0d2910a&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=c855898533554ff5b216819dc74c80e8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.291042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.395374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659bb2-6c55-4510-a8d5-af2f55d779aa None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.455566 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.456314 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.457081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.457362 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.457636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.458447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:34.458753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.461214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.470972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 61/129] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.472417 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.498153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.511874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373433b-02af-49f1-a010-514a66b80c7f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.567796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.568484 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.568766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.569041 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/9134c5ce0660415e9a3bc4b4ecb08e82/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.573207 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb86611f7b4363a6d822a20268eb19, user_id=9134c5ce0660415e9a3bc4b4ecb08e82, role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.574765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.624575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49ceb8-de5d-41c6-806f-28a258b1f5dc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.661442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.662150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.662440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.662777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.663542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:34.664862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.665230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.673913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 62/130] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.674839 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.702914 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 69/131] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:34 2022] PUT /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/9134c5ce0660415e9a3bc4b4ecb08e82/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:34.704997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.772926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.794428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b730-6199-462a-b22e-1ac0f3ab1b94 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.862201 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.863376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.863667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.863950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.865497 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.867005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.869204 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 63/132] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.874854 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.969392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b060c-f5f5-443b-9cc6-760b33c1cfe2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.035282 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.036178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.036465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.036745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.038069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.039700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.041390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 70/133] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:35.046898 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:35.052690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1928e-a19f-48b6-8748-696ad173cc3f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.059568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.128400 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.129210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.129495 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.129766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] PATH_INFO: `/v3/projects/77837403f948450491b7b0fcaf459861` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.135167 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] RBAC: Authorizing `identity:get_project(project_id=77837403f948450491b7b0fcaf459861)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.136804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.138920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 64/134] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/projects/77837403f948450491b7b0fcaf459861 => generated 312 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:35.145048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:35.164088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2573dc-3c31-4732-86c3-1692dada55bb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.195105 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.199890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.200819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.201111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.201386 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.202641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.204150 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.206599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 71/135] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:35 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:35.211999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:35.308670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.319107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6805d-502e-43b0-a420-e4957272c709 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.362535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8b742-011c-4029-9655-927ece760a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.363450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8b742-011c-4029-9655-927ece760a46 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.363842 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8b742-011c-4029-9655-927ece760a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.364128 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8b742-011c-4029-9655-927ece760a46 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/2ba410fb97cd412fa30f98b46a43dd28/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.371924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8b742-011c-4029-9655-927ece760a46 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb86611f7b4363a6d822a20268eb19, user_id=2ba410fb97cd412fa30f98b46a43dd28, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.373589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8b742-011c-4029-9655-927ece760a46 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.401743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1bc03-17d4-4a15-9dfb-12ee218b2daf None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.452715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef693a3f-16b5-44a3-a2e7-18c63fb89b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.453613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef693a3f-16b5-44a3-a2e7-18c63fb89b35 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.453891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef693a3f-16b5-44a3-a2e7-18c63fb89b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.454189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef693a3f-16b5-44a3-a2e7-18c63fb89b35 None admin] PATH_INFO: `/v3/projects/77837403f948450491b7b0fcaf459861/users/9134c5ce0660415e9a3bc4b4ecb08e82/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.456701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef693a3f-16b5-44a3-a2e7-18c63fb89b35 None admin] RBAC: Authorizing `identity:create_grant(project_id=77837403f948450491b7b0fcaf459861, user_id=9134c5ce0660415e9a3bc4b4ecb08e82, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.458187 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef693a3f-16b5-44a3-a2e7-18c63fb89b35 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.463303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 72/136] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:35 2022] PUT /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/2ba410fb97cd412fa30f98b46a43dd28/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:42:35.560118 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 65/137] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:35 2022] PUT /identity/v3/projects/77837403f948450491b7b0fcaf459861/users/9134c5ce0660415e9a3bc4b4ecb08e82/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:36.133396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e873b9-0b2e-49a5-8f22-1a5211a345bc None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.134099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e873b9-0b2e-49a5-8f22-1a5211a345bc None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.134099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e873b9-0b2e-49a5-8f22-1a5211a345bc None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.135357 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 73/138] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:36.140687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.141179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.141330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.159704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:36.160899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:36.186253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6374-6dc3-450c-95fd-dc31c677ea04 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.186541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6374-6dc3-450c-95fd-dc31c677ea04 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.186801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6374-6dc3-450c-95fd-dc31c677ea04 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.188430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 74/139] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:36.206513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.206807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.207084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.252226 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:36.253982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:36.272302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 66/140] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:36.312559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:36.326948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 75/141] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:36.334378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:36.377395 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:36.510800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:36.521059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae6b5-c272-405c-a7ee-1481e7a1950f None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:36.588567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442c524-275d-486d-84d6-541bf41118c9 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:36.618097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:36.618929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.619222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.619503 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.628783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:36.629585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.629870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.630186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.631381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:36.632823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:36.634510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 76/142] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:36.640135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:36.652457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:36.660704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:36.661526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:36.662340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:36.666713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 67/143] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:36.670976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:36.785696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c1a7-b0b1-4f0b-b696-2fc63d5ca667 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:36.811963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:36.829436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:36.830146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.830429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.830704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.831995 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:36.833439 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:36.855319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:36.899222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e68f-428b-4957-ae14-f8f11a1ee718 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:36.941427 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:36.942181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:36.942471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:36.942755 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:36.943509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:36.943817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:36.945270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:36.954295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 68/144] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:36.962961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.017738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 77/145] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:36 2022] POST /identity/v3/users => generated 283 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:37.094456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012865-f9a7-45b6-b7a2-fddda27eb968 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.142588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:37.145486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b77553-fcd5-42bb-baf2-7745e152913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.146379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b77553-fcd5-42bb-baf2-7745e152913d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.147076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b77553-fcd5-42bb-baf2-7745e152913d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.147362 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b77553-fcd5-42bb-baf2-7745e152913d None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.150874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b77553-fcd5-42bb-baf2-7745e152913d None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:37.155961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b77553-fcd5-42bb-baf2-7745e152913d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:37.155961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 69/146] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:36 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:37.161056 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.310197 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545081bc-8032-4430-902e-a1d27d51c6c0 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.347534 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.348125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.348414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.348688 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.410274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc5bcb-81a8-4fd8-9f80-c2f46990a944 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.410920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc5bcb-81a8-4fd8-9f80-c2f46990a944 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.411397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc5bcb-81a8-4fd8-9f80-c2f46990a944 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.412993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 70/147] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:37.419607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.420157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.420635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.434548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:37.454916 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:37.456674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:37.457731 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:37.459135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:37.582415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 71/148] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:37.606622 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.631579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 78/149] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:37.638213 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.740454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f725e-dc8c-47b6-bf1e-efa3f753d129 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.760231 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:37.771267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48497f3a-e163-426d-89ff-0b10d8761c7c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.778019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:37.789843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07bef-48c4-4ef6-a343-22460d68f940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.790688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07bef-48c4-4ef6-a343-22460d68f940 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.790970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07bef-48c4-4ef6-a343-22460d68f940 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.791246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07bef-48c4-4ef6-a343-22460d68f940 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.807497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20e742-04f5-420d-aea3-be33c3a0c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.808381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20e742-04f5-420d-aea3-be33c3a0c68f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.808673 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20e742-04f5-420d-aea3-be33c3a0c68f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.808952 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20e742-04f5-420d-aea3-be33c3a0c68f None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.809116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07bef-48c4-4ef6-a343-22460d68f940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:37.810210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20e742-04f5-420d-aea3-be33c3a0c68f None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:37.811679 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20e742-04f5-420d-aea3-be33c3a0c68f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:37.813675 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 79/150] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:37.822252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0e06-56fa-4c94-9acb-58070ed15da9 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.822252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0e06-56fa-4c94-9acb-58070ed15da9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.822252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0e06-56fa-4c94-9acb-58070ed15da9 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.822252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 80/151] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-85d07bef-48c4-4ef6-a343-22460d68f940 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:37.822609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:37.823378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:37.824127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 72/152] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:37.824757 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0e06-56fa-4c94-9acb-58070ed15da9 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.824824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.825082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.826325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.845888 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:37.849219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:37.939691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0e06-56fa-4c94-9acb-58070ed15da9 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.941492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 73/153] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:37.943206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.983169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.983840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.984122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.984458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.985310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=9134c5ce0660415e9a3bc4b4ecb08e82) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:37.985621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:37.987113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:37.994506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 81/154] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/role_assignments?user.id=9134c5ce0660415e9a3bc4b4ecb08e82&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:37.995338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.035114 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.115073 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416eea-24aa-45c5-a72c-a05fe40a9699 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.141116 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cc249-7519-4a60-a914-7d84575c13a7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.143948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.179508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.180214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.180498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.180779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.181557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:38.181867 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.183308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.198045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 74/155] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:38.199267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.200189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.200476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.200755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.215126 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.219380 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:38.229263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:38.230458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:38.231373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:38.235299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 82/156] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:38.239929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.347740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.371200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd248fa-5e94-404a-9837-2d4f57db8e2b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.412744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.413769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.414240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.414658 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.415559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:38.416006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.417557 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.429872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05326f2-6dbb-4a99-83ba-e8999d6f8e73 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.430484 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 83/157] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:38.437525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.480844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.481659 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.481947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.482275 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] PATH_INFO: `/v3/users/0496d401474c4cf9af323e0fd0d2910a` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.486843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] RBAC: Authorizing `identity:get_user(user_id=0496d401474c4cf9af323e0fd0d2910a)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.488406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.490574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 75/158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/users/0496d401474c4cf9af323e0fd0d2910a => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:38.505518 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.573613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818c1b-8d2c-44b3-96f0-527535109485 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.598069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.621088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.622260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.622760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.623173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.624565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.626199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.628141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 84/159] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:38.634046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.656986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.684032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d2f4-d546-424d-ab00-4e467a855262 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.799778 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deac100-4507-4d2f-ad01-80e50f9cce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.801239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deac100-4507-4d2f-ad01-80e50f9cce4a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.801540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deac100-4507-4d2f-ad01-80e50f9cce4a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.802164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deac100-4507-4d2f-ad01-80e50f9cce4a None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.810418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deac100-4507-4d2f-ad01-80e50f9cce4a None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.810418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deac100-4507-4d2f-ad01-80e50f9cce4a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.810418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 76/160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:38.816379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395d658-9f90-4e84-a69e-052e70ecc343 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.816673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395d658-9f90-4e84-a69e-052e70ecc343 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.816952 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395d658-9f90-4e84-a69e-052e70ecc343 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.818334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 77/161] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:38.819160 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395d658-9f90-4e84-a69e-052e70ecc343 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.845002 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112f001-e9ec-47f5-97a8-bd8b860ee646 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.885278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec8615-b07a-4015-bd70-e124dc015106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.886222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec8615-b07a-4015-bd70-e124dc015106 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.886650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec8615-b07a-4015-bd70-e124dc015106 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.887076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec8615-b07a-4015-bd70-e124dc015106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.971433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec8615-b07a-4015-bd70-e124dc015106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.973728 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec8615-b07a-4015-bd70-e124dc015106 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.991794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.052382 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395d658-9f90-4e84-a69e-052e70ecc343 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.080315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.085739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 85/162] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:39.088330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c2e5-781e-46be-bb46-eca758283df5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.089686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c2e5-781e-46be-bb46-eca758283df5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.089980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c2e5-781e-46be-bb46-eca758283df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.112108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7db511-7ca1-4d72-adbb-062f5f5d7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.113387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7db511-7ca1-4d72-adbb-062f5f5d7a0d None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.113977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7db511-7ca1-4d72-adbb-062f5f5d7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.114449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7db511-7ca1-4d72-adbb-062f5f5d7a0d None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/0496d401474c4cf9af323e0fd0d2910a/roles/c855898533554ff5b216819dc74c80e8` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.117298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7db511-7ca1-4d72-adbb-062f5f5d7a0d None admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb86611f7b4363a6d822a20268eb19, user_id=0496d401474c4cf9af323e0fd0d2910a, role_id=c855898533554ff5b216819dc74c80e8)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.118974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7db511-7ca1-4d72-adbb-062f5f5d7a0d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.127109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-5743c2e5-781e-46be-bb46-eca758283df5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:39.129152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-5743c2e5-781e-46be-bb46-eca758283df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:39.197390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 78/163] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:38 2022] PUT /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19/users/0496d401474c4cf9af323e0fd0d2910a/roles/c855898533554ff5b216819dc74c80e8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:39.199999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95400ab7-d109-42ac-90ed-a975a506e576 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.200485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95400ab7-d109-42ac-90ed-a975a506e576 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.200939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95400ab7-d109-42ac-90ed-a975a506e576 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.203996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 79/164] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:39.205738 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95400ab7-d109-42ac-90ed-a975a506e576 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.229472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 86/165] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:39.231570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.232089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.232611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.253414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:39.255209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:39.298515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.335802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 87/166] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:39.337096 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.439903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95400ab7-d109-42ac-90ed-a975a506e576 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.451447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658ca9-8ca4-43ec-b759-d616069a0e12 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.490607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.491886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.492489 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.493239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.505512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:39.514238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:39.515106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:39.516103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:39.516690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.516690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.516690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.516690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.517451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.518686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 88/167] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:39.519498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.520366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.521602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 80/168] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.526984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.558800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.584487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.634124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abc2ac-c0bf-4d42-b124-53891eb350d7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.640597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2673b9-1510-4149-b006-c1c76d84c648 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.673999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.675158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.675739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.676298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.687705 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:39.696911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:39.697984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:39.699049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:39.699783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.699783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.699783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.700025 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.700677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:39.700978 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.702074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 89/169] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:39.702381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.703286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.725685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 81/170] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.726215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.812317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322c0ee-0676-469e-8611-61cd59ab667f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.814529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.857104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.858027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.858477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.858894 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.859877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] RBAC: Adding query filter params (role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=9134c5ce0660415e9a3bc4b4ecb08e82) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:39.860425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.864103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.872254 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 90/171] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/role_assignments?user.id=9134c5ce0660415e9a3bc4b4ecb08e82&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.873278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.896147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9668d-72a6-4f33-87f7-584927e27907 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.908890 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.947317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.948015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.948300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.948574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.949356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:39.949659 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.951148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.964440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 82/172] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.969978 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.999195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872adbb-02d4-431f-8ea7-ee0e5e4ee1ce None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.064300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.065588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.066736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.067161 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] PATH_INFO: `/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.071159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] RBAC: Authorizing `identity:get_user(user_id=9134c5ce0660415e9a3bc4b4ecb08e82)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.072939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.075623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 91/173] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/users/9134c5ce0660415e9a3bc4b4ecb08e82 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:40.081160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.091345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256af42-dfca-4766-89b0-f7c667799ed3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.126762 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.127200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.127644 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.127923 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.129049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] PATH_INFO: `/v3/users/2ba410fb97cd412fa30f98b46a43dd28` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.130028 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] RBAC: Authorizing `identity:get_user(user_id=2ba410fb97cd412fa30f98b46a43dd28)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.131600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.133349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 83/174] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/users/2ba410fb97cd412fa30f98b46a43dd28 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:40.138989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.181181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.280278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd2ef-585a-4cf1-afe9-51caee1bbd4a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.317283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fe86d-9751-4f63-bed3-f93a4719ae41 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.367183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.368144 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.368733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.369527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.374137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.374790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.375071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.375342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.511696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.514429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.517464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.519162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.567762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.595175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.704483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 84/175] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:40.710270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.762411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 92/176] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:40.763265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.828035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5df7-9cc0-4e4c-af41-8bf909137b20 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.876626 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.877581 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.877868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.878169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.879696 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.881205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.883233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 85/177] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:40.900014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.909635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.945354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7f9f-e345-4560-9da3-e6441611f99a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.970406 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.981018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.981966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.982334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.982662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] PATH_INFO: `/v3/projects/77837403f948450491b7b0fcaf459861` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.983923 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] RBAC: Authorizing `identity:get_project(project_id=77837403f948450491b7b0fcaf459861)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.985572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.987343 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 93/178] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/projects/77837403f948450491b7b0fcaf459861 => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:40.993165 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.096492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4942-4040-4cd1-a831-31eb0d51a045 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.106213 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41b0b0-bebf-4e08-a256-4922813978dd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.157054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.158108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.158585 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.159079 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.160230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=2ba410fb97cd412fa30f98b46a43dd28) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:41.160833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.168051 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a50910-e8f1-465e-8ed8-75dfc0e74350 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.174845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.175576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.175862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.176137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.182033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=77837403f948450491b7b0fcaf459861, user.id=9134c5ce0660415e9a3bc4b4ecb08e82) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:41.182033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.182033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b267-c098-4510-806b-1cafe7b8852d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.187289 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 86/179] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/role_assignments?user.id=2ba410fb97cd412fa30f98b46a43dd28&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.193366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 94/180] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/role_assignments?user.id=9134c5ce0660415e9a3bc4b4ecb08e82&scope.project.id=77837403f948450491b7b0fcaf459861&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.324434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330fdc-475f-47d8-af65-f29667fd7d17 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.325727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330fdc-475f-47d8-af65-f29667fd7d17 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.326483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330fdc-475f-47d8-af65-f29667fd7d17 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.329533 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 87/181] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:41.336245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f00369-5d66-4245-898b-7da78b8c0636 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.336752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f00369-5d66-4245-898b-7da78b8c0636 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.336825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f00369-5d66-4245-898b-7da78b8c0636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.363385 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-83f00369-5d66-4245-898b-7da78b8c0636 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:41.364536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-83f00369-5d66-4245-898b-7da78b8c0636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:41.440519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.462404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 95/182] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:41.509995 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330fdc-475f-47d8-af65-f29667fd7d17 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.597543 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.636330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb403d-baf5-49f6-924f-ddbb357cfb74 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.636617 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb403d-baf5-49f6-924f-ddbb357cfb74 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.636858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb403d-baf5-49f6-924f-ddbb357cfb74 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.638365 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 96/183] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:41.643898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330fdc-475f-47d8-af65-f29667fd7d17 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.644408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.644697 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.644976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.673429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:41.677638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:41.693511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.694435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.694717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.695040 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.713173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:41.724222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:41.725121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:41.726325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 88/184] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:41.731543 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.765283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 97/185] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:41.791101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.851331 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7367f-814d-4b2e-96ae-41dab97fea32 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.881165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.894031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.894031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.894031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.894031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.894395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:41.894395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.894395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccc5f-f429-40f8-835a-546e3fea162f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.894582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7144c4-6f74-4907-b257-4ccf3dbc9354 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.903517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 89/186] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:41.905485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457b5c7-e3f8-4aa7-843a-425560ce5c9d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.905785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457b5c7-e3f8-4aa7-843a-425560ce5c9d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.906092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457b5c7-e3f8-4aa7-843a-425560ce5c9d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.911883 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 90/187] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:41.911883 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457b5c7-e3f8-4aa7-843a-425560ce5c9d None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.918401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.937752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fe2df-031c-49d1-80c4-bfe0714a4cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.938584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fe2df-031c-49d1-80c4-bfe0714a4cd1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.939124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fe2df-031c-49d1-80c4-bfe0714a4cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.939403 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fe2df-031c-49d1-80c4-bfe0714a4cd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.940570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fe2df-031c-49d1-80c4-bfe0714a4cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.942028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fe2df-031c-49d1-80c4-bfe0714a4cd1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.943806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 98/188] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.945082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.945383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.945639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.964714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:41.965944 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:42.024866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457b5c7-e3f8-4aa7-843a-425560ce5c9d None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.039200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 99/189] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:42.051518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.061210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.062120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.062399 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.062672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] PATH_INFO: `/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.077277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=cd4454f3575c4a22a8aea53ffe9a25c8)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.078766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.080453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 91/190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:42.081206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.139639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.171469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e0a2f-c05b-487e-b150-afe9deab617c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.186294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.199704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdabf-ef8a-49ba-8ead-c52db9384ca0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.208744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.209425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.209708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.209986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.211106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.213029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.236381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.237192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.237472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.240259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.248375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:42.257091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:42.258150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:42.259070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 92/191] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:42.259792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.352426 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 100/192] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:42 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:42.353266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.360384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe56ad-6f8a-4018-90de-4efa258c5372 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.371107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.400943 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ab237-01a6-4d4b-8549-88cfabb3f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.401623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab237-01a6-4d4b-8549-88cfabb3f113 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.401903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab237-01a6-4d4b-8549-88cfabb3f113 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.402212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab237-01a6-4d4b-8549-88cfabb3f113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.469366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2afa6-552f-4ba7-9680-0493890adf12 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.481402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ab237-01a6-4d4b-8549-88cfabb3f113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.482911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ab237-01a6-4d4b-8549-88cfabb3f113 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.510341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.511186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.511475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.511750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.512518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:42.512840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.514245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17296e0-7115-435e-88bd-524191b18b5c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.528483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 101/193] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.528483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0b19e-75fd-48d0-aa52-4a7da71fe02e None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.528483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0b19e-75fd-48d0-aa52-4a7da71fe02e None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.528483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0b19e-75fd-48d0-aa52-4a7da71fe02e None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.528483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 102/194] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:42.530309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0b19e-75fd-48d0-aa52-4a7da71fe02e None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.569496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.613867 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 93/195] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:42.614579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.615734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.616021 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.639487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:42.641076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:42.646432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.675689 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0b19e-75fd-48d0-aa52-4a7da71fe02e None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.721025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.721835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.722197 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.722497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] PATH_INFO: `/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.723886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] RBAC: Authorizing `identity:get_user(user_id=cd4454f3575c4a22a8aea53ffe9a25c8)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.725341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.731849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 103/196] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:42.732886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.745365 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 94/197] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:42.746713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.868173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.871309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae9f9-bf7a-4d3a-b9e0-d6d78ad35d46 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.881841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91af42-2435-45de-a23e-44e0fab5a870 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.907080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.908045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.908653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.909044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.910470 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.912026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.913867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 104/198] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.914825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.927018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.927739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.928015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.928702 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.940788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.023490 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.026141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.027794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8b35a-b252-4ed4-8444-f1acabd0df5b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.069373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.070516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.070902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.071330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.084977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:43.094877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:43.096302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:43.097866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:43.099711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 105/199] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:43.100453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.145221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.205277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 95/200] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:43.206818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.242661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948e40f-beb1-4d7c-98a9-63c677988781 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.253624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.288972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.290914 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.291364 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.291781 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.293551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=cd4454f3575c4a22a8aea53ffe9a25c8) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:43.294469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.296366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.307893 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 106/201] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/role_assignments?user.id=cd4454f3575c4a22a8aea53ffe9a25c8&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.309630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.333605 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3ed74-7049-4782-abde-50579542fd6a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.430841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.432390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.432878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.433350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.434375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:43.434893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.436563 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.456218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 96/202] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.464869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.482517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.496905 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2142fc-b097-4ebc-b841-9b0036881e75 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.538288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.539218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.548630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.548960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.551339 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.552812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.557928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 107/203] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.561366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.707204 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.710428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cecb0-ac52-456e-8b8f-8923b2a37c00 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.733886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634902f-3700-415e-8b44-cbbcd535a277 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.771627 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.772522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.772806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.773078 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] PATH_INFO: `/v3/users/0496d401474c4cf9af323e0fd0d2910a` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.774848 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] RBAC: Authorizing `identity:get_user(user_id=0496d401474c4cf9af323e0fd0d2910a)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.776330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.777989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 97/204] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/users/0496d401474c4cf9af323e0fd0d2910a => generated 289 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:43.782836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.783525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.783804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.784076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.784972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] RBAC: Adding query filter params (role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=cd4454f3575c4a22a8aea53ffe9a25c8) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:43.785282 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.786811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.789858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.806170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 108/205] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/role_assignments?user.id=cd4454f3575c4a22a8aea53ffe9a25c8&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.911429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58c631-7ed1-4a93-9692-b5add7d16289 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.948404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.948994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.949278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.949551 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.986488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:44.021925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:44.023465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:44.142915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 98/206] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:44.148987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.185711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:44.287296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046b291-f4a5-42ca-b47f-5a6415f6a97d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:44.338755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be41f61-afd1-4346-bbfd-aabd0c5bef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:44.339979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be41f61-afd1-4346-bbfd-aabd0c5bef3e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.340509 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be41f61-afd1-4346-bbfd-aabd0c5bef3e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.340986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be41f61-afd1-4346-bbfd-aabd0c5bef3e None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.342439 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be41f61-afd1-4346-bbfd-aabd0c5bef3e None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:44.344158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be41f61-afd1-4346-bbfd-aabd0c5bef3e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:44.346408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 109/207] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:44 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:44.352509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.487508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:44.531663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b606-21fa-483f-96b9-104326fb77f4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:44.584389 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:44.585066 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.585404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.585687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.586582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] RBAC: Adding query filter params (role.id=c855898533554ff5b216819dc74c80e8, scope.project.id=3cbb86611f7b4363a6d822a20268eb19, user.id=0496d401474c4cf9af323e0fd0d2910a) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:44.586897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:44.588349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cee03-813b-4dad-87be-53f972b3245f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:44.599085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 99/208] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:44 2022] GET /identity/v3/role_assignments?user.id=0496d401474c4cf9af323e0fd0d2910a&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=c855898533554ff5b216819dc74c80e8 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:44.771858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144f50f-75f3-48a3-bce9-49159b0a6d53 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.772917 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144f50f-75f3-48a3-bce9-49159b0a6d53 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.773423 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144f50f-75f3-48a3-bce9-49159b0a6d53 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.775154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 110/209] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:44.780971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.781554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.782053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.806383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:44.807990 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:44.905785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 100/210] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:44.928160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4082c-3327-4a95-a755-9ab64a264ca3 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.928748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4082c-3327-4a95-a755-9ab64a264ca3 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.929241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4082c-3327-4a95-a755-9ab64a264ca3 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.931075 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.931328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 111/211] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:44.936507 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.937033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.937507 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.958910 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:44.960961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:45.003364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.052411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 112/212] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:45.060545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3227d36-38cf-4aed-83fe-297fee5f2cf9 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.081386 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.112936 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.114404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.114754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.115929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.116485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.117938 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.119783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.121898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 101/213] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:45.131443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.206562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45e63b-9305-48c0-9105-22d6a622223a None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.247985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.249182 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.249670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.250172 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.262531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:45.267574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03671855-fc35-4d85-8acc-16e29f54ecfe None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:45.271413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:45.272294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:45.273081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:45.275530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 113/214] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:45.281583 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.298242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.309610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.309610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.309610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.309610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.310032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.310032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.376766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.396105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3687c-d741-4657-8afa-f0d5d32c4426 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.432725 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 102/215] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:45 2022] POST /identity/v3/users => generated 297 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:45.443932 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.444736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.445026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.445450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.446308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:45.446669 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.448187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.458627 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 114/216] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:45.465101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.586908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad199-4f88-487d-81c4-233d7003d888 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.624450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4fa60-cd1d-4176-8595-58851136500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.625337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4fa60-cd1d-4176-8595-58851136500a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.625620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4fa60-cd1d-4176-8595-58851136500a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.625899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4fa60-cd1d-4176-8595-58851136500a None admin] PATH_INFO: `/v3/users/b08ba4e8116144578e013234d3271618` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.640116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4fa60-cd1d-4176-8595-58851136500a None admin] RBAC: Authorizing `identity:get_user(user_id=b08ba4e8116144578e013234d3271618)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.641600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4fa60-cd1d-4176-8595-58851136500a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.643520 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 103/217] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:45 2022] GET /identity/v3/users/b08ba4e8116144578e013234d3271618 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:45.649004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.763080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b93ec-9fe4-47ec-8a87-a6e233574fc4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.807335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.808819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.809100 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.809376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.829200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.866210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a27e-264b-405a-a7d2-bebd85881fa8 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.866893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a27e-264b-405a-a7d2-bebd85881fa8 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.867373 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a27e-264b-405a-a7d2-bebd85881fa8 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.869260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 104/218] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:45.874617 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.875212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.875594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.896787 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.898474 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.900477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:45.901680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:45.949728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.016049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 105/219] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:46.023442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 115/220] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:46.028575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.040321 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.126809 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.195367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090906ab-079e-4338-908c-b2525d2ec3c4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.200655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2a665-87b4-4756-915c-56854d589e20 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.261634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.262467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.262748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.263040 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.275537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:46.285317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:46.286228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:46.286993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 116/221] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:46.294038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.294904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.295015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.295217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.295494 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.297033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.298552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.300472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 106/222] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.307008 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.366423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.410130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad556a-1ec9-42eb-b4fa-e86e24c3c8cb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.413932 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.435413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699d44-51e5-4840-839f-6ed77083684a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.454381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.455151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.457962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.457962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.457962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:46.458371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.459797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64194446-d762-4736-b7e4-81a33ac5563e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.472412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 117/223] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.473634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10eaa0-51ef-4fe6-8e77-2076b4ae68fe None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.473995 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10eaa0-51ef-4fe6-8e77-2076b4ae68fe None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.474373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10eaa0-51ef-4fe6-8e77-2076b4ae68fe None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.475708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 118/224] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:46.477466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.478488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.479633 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.480932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.481088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10eaa0-51ef-4fe6-8e77-2076b4ae68fe None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.482373 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=b08ba4e8116144578e013234d3271618) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:46.482874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.484507 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc3ea6-d2e8-4bab-92a2-1a7ef500c41f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.491217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 107/225] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/role_assignments?user.id=b08ba4e8116144578e013234d3271618&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.500116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.500705 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.501272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.535788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:46.537354 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:46.618040 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10eaa0-51ef-4fe6-8e77-2076b4ae68fe None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.656151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.657049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.657335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.657606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] PATH_INFO: `/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.664226 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] RBAC: Authorizing `identity:get_user(user_id=cd4454f3575c4a22a8aea53ffe9a25c8)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.665682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.673838 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 119/226] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.679973 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.738968 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.792106 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 108/227] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:46.803098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde4871-7d9e-4dfa-9bf6-7969218bee2f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.838882 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.839800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.840126 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.840410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.841626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.843189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.844981 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 120/228] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:46.846150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.854989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.886645 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.989112 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5465-a97d-4cea-a9fc-463949ce886d None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.030637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35ca7-9ff6-45a9-bea6-5ef111d1d7c1 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.032389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.070772 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.071834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.072573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.073938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/cd4454f3575c4a22a8aea53ffe9a25c8/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.077860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.078250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] RBAC: Authorizing `identity:create_grant(project_id=ae63c1ef62974692a21343d5415af4fe, user_id=cd4454f3575c4a22a8aea53ffe9a25c8, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.080288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.080753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.081149 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.081522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.096820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:47.106299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:47.109451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:47.110268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 109/229] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:47.114361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.167297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 121/230] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:46 2022] PUT /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/cd4454f3575c4a22a8aea53ffe9a25c8/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:47.223809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.230903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608194c-574b-4c64-839d-f4a2841854d6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.270224 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.271291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.271579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.271859 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.272674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:47.272980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.274600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.284890 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 110/231] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:47.291187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.429833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79506793-feb9-4415-a212-26a58f20d72a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.467709 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.468598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.468882 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.469147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] PATH_INFO: `/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.470402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] RBAC: Authorizing `identity:get_user(user_id=cd4454f3575c4a22a8aea53ffe9a25c8)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.471854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.478119 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 122/232] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:47.480329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.632817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703eeea7-90da-47b7-a61e-9ab75e2fd8e3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.679711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2ea7f-446e-4ade-87bb-c1f5e3184f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.680595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2ea7f-446e-4ade-87bb-c1f5e3184f48 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.680881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2ea7f-446e-4ade-87bb-c1f5e3184f48 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.681153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2ea7f-446e-4ade-87bb-c1f5e3184f48 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.682411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2ea7f-446e-4ade-87bb-c1f5e3184f48 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.683901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2ea7f-446e-4ade-87bb-c1f5e3184f48 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.685678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 111/233] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:47.692413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.742391 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.812955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693ae5e-47f5-47e3-bccc-2cd39d9f176b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.854960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90671-86a5-4cc7-a63c-c9f0cb693df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.856648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90671-86a5-4cc7-a63c-c9f0cb693df6 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.857130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90671-86a5-4cc7-a63c-c9f0cb693df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.857601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90671-86a5-4cc7-a63c-c9f0cb693df6 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/cd4454f3575c4a22a8aea53ffe9a25c8/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.860167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90671-86a5-4cc7-a63c-c9f0cb693df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae63c1ef62974692a21343d5415af4fe, user_id=cd4454f3575c4a22a8aea53ffe9a25c8, role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.861853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90671-86a5-4cc7-a63c-c9f0cb693df6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.981896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 123/234] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:47 2022] PUT /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/cd4454f3575c4a22a8aea53ffe9a25c8/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:48.022002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1232ec-5c14-4609-9c14-996737c849fa None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.022002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1232ec-5c14-4609-9c14-996737c849fa None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.022002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1232ec-5c14-4609-9c14-996737c849fa None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.022002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 112/235] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:48.028726 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006bad9-fc5d-4e42-965d-d61c9749f192 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.029349 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006bad9-fc5d-4e42-965d-d61c9749f192 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.029830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006bad9-fc5d-4e42-965d-d61c9749f192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.050556 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-7006bad9-fc5d-4e42-965d-d61c9749f192 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:48.052704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-7006bad9-fc5d-4e42-965d-d61c9749f192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:48.136200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 124/236] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:48.161313 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1232ec-5c14-4609-9c14-996737c849fa None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.174635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9397bd4-771b-4117-948d-e467172ea976 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.175296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9397bd4-771b-4117-948d-e467172ea976 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.178155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9397bd4-771b-4117-948d-e467172ea976 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.181652 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 125/237] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:48.185642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.186260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.186756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.221872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.231690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:48.235286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:48.259383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.318348 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1232ec-5c14-4609-9c14-996737c849fa None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.321347 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 126/238] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:48.347198 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.358029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.359265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.359846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.360154 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.374366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:48.384973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:48.386123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:48.387244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 113/239] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:48.393155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.486211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d67e4-5564-4389-b3e1-0590682241bd None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.560934 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.566749 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.568524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.568524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.568847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.574813 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.576289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.577728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c018fbc-18ab-4610-8cc7-d0b33482c900 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.578313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 127/240] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:48.592277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.625990 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.626712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.626992 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.627257 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.628014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:48.630185 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.631649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.648524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.652248 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 114/241] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:48.657860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.820778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7898dc-bd5e-433c-a300-31d0c7a4b45c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.829169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de643-5778-4388-af0c-24034c6efdc3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.874304 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1d60-e136-4d9c-a985-c060c516e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.875511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1d60-e136-4d9c-a985-c060c516e437 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.875933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1d60-e136-4d9c-a985-c060c516e437 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.876387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1d60-e136-4d9c-a985-c060c516e437 None admin] PATH_INFO: `/v3/users/421b09a23b124ab38ae1c39b23815300` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.885236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb514c-a6f8-4eeb-af78-97ba68c9995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.885919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb514c-a6f8-4eeb-af78-97ba68c9995c None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.886245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb514c-a6f8-4eeb-af78-97ba68c9995c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.886525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb514c-a6f8-4eeb-af78-97ba68c9995c None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.887241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb514c-a6f8-4eeb-af78-97ba68c9995c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.890517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb514c-a6f8-4eeb-af78-97ba68c9995c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.899224 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1d60-e136-4d9c-a985-c060c516e437 None admin] RBAC: Authorizing `identity:get_user(user_id=421b09a23b124ab38ae1c39b23815300)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.901736 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1d60-e136-4d9c-a985-c060c516e437 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.904137 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 115/242] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/users/421b09a23b124ab38ae1c39b23815300 => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:48.905687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddb0af-f1a8-4008-8041-6c7aa1c73050 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.906105 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddb0af-f1a8-4008-8041-6c7aa1c73050 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.906264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddb0af-f1a8-4008-8041-6c7aa1c73050 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.907675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 116/243] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:48.909458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddb0af-f1a8-4008-8041-6c7aa1c73050 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.918833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.009401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddb0af-f1a8-4008-8041-6c7aa1c73050 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.043076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.043911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.044076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.044322 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.062504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 128/244] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:48 2022] POST /identity/v3/users => generated 293 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:49.064938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.065999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.066885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.115700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:49.117076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:49.182935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.184429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.204315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.261523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 129/245] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:49.292013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.295246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.339345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 117/246] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:49.346109 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.416919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa36a1-789a-43ba-be2f-9e9e5f4c418b None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.456243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.457555 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.458088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.458581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.472416 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5acbc-b85b-44f5-9354-3b57abfb2bbb None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.482525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.483980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:49.493039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:49.500296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:49.501069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 130/247] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:49.503638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.517718 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.518635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.518909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.519181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.520434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.521882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.523856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 118/248] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:49.529732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.587432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.632610 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aaac6-3a56-474c-8868-393b43e62669 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.654492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d104-c260-4d05-b808-a11d3e50a3c1 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.680238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.680929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.681217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.681488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.682277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:49.682594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.684066 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398099-a046-41cd-8575-dd1a2ee52a68 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.694205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.695219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.695706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.696203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.697316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] RBAC: Adding query filter params (role.id=c855898533554ff5b216819dc74c80e8, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=421b09a23b124ab38ae1c39b23815300) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:49.697845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.698295 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 131/249] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:49.699629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d3a4-f564-48a5-b1de-aef2d9587588 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.699915 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d3a4-f564-48a5-b1de-aef2d9587588 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.700192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d3a4-f564-48a5-b1de-aef2d9587588 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.700462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ca41-ceba-453c-b911-cd9f85cbcdc4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.701459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 132/250] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:49.707113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d3a4-f564-48a5-b1de-aef2d9587588 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.708157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 119/251] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/role_assignments?user.id=421b09a23b124ab38ae1c39b23815300&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=c855898533554ff5b216819dc74c80e8 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:49.714003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.714599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.715054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.735834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:49.737301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:49.780901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.829842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 120/252] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:49.854534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.856650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d3a4-f564-48a5-b1de-aef2d9587588 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.926506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.941782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.942761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.943043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.943317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] PATH_INFO: `/v3/users/b08ba4e8116144578e013234d3271618` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.944671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=b08ba4e8116144578e013234d3271618)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.948301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.950094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 133/253] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/users/b08ba4e8116144578e013234d3271618 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:49.964219 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.047075 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eaf320-bdda-408e-8f44-a5419fb3b0ec None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.130840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.132057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.132566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.133040 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.142044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431642a-b8da-462a-9ba3-1d33eb62b1c6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.157388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:50.176317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:50.177315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:50.178131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:50.180588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 121/254] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:50.191151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.195692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.196911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.196911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.197109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.202879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.204357 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.206633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 134/255] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.214796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.249958 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.308366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.348795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce29a0-07c5-46b6-8cc3-060f1c9e8589 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.365303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f6820-914e-4b03-badd-733f00eb9bce None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.384523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.385509 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.385793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.386103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/b08ba4e8116144578e013234d3271618/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.388199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae63c1ef62974692a21343d5415af4fe, user_id=b08ba4e8116144578e013234d3271618, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.389766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.419183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.420409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.420875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.421328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.422487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:50.422793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.424182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.434036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 122/256] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.439590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.479135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 135/257] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:50 2022] PUT /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/b08ba4e8116144578e013234d3271618/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:50.578470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.591347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087207-985e-4bf7-81ba-9c36d01ebdb2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.638987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44546da2-dc9c-4aa3-9048-f3dc93ca42d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.639958 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546da2-dc9c-4aa3-9048-f3dc93ca42d8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.640639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546da2-dc9c-4aa3-9048-f3dc93ca42d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.640639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44546da2-dc9c-4aa3-9048-f3dc93ca42d8 None admin] PATH_INFO: `/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.641931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44546da2-dc9c-4aa3-9048-f3dc93ca42d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cd4454f3575c4a22a8aea53ffe9a25c8)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.643446 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44546da2-dc9c-4aa3-9048-f3dc93ca42d8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.646258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 123/258] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.652070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.815753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a50bd-7d8b-4750-865f-e654153cdd27 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.840168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.848268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49059044-5875-4e4c-a5b8-5ae4846d2284 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.848604 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49059044-5875-4e4c-a5b8-5ae4846d2284 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.848852 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49059044-5875-4e4c-a5b8-5ae4846d2284 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.854640 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 124/259] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:50.855849 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.856139 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.856414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.857782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.858473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.858756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.859031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.907029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:50.908230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:50.963484 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.964944 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.018347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 125/260] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:51.053740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.194062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 136/261] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:51.198299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.254400 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.257927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.305355 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0b80c-d3a3-4a22-8ca0-46da7eae26c2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.428674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25f81-cd5a-4808-b8b3-fe93a7e6c5ba None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.451709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.452465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.452751 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.453027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.482875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:51.491887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:51.492851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:51.493639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:51.498477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 126/262] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:51.498715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.499608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.499893 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.500176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.501419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.502959 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.503868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.504810 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 137/263] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:51.511728 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.764108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.777577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2bd88-877f-453e-8980-39ac0f7ba293 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.796216 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a33642-fbf8-41aa-83af-55b6e4605b8c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.832191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.832926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.833699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.834163 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.835058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:51.835525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.837047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.854057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 127/264] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:51.859475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.882702 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.888705 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.889379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.889664 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.889944 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.890815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=cd4454f3575c4a22a8aea53ffe9a25c8) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:51.891125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.892571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.911635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 138/265] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/role_assignments?user.id=cd4454f3575c4a22a8aea53ffe9a25c8&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:52.020328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833acce-9ec4-4636-b68e-de9848ea4892 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.073653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5bea2-8b23-40fb-ba77-81e9e3a71054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.074620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bea2-8b23-40fb-ba77-81e9e3a71054 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.074907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bea2-8b23-40fb-ba77-81e9e3a71054 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.075187 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bea2-8b23-40fb-ba77-81e9e3a71054 None admin] PATH_INFO: `/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.076471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5bea2-8b23-40fb-ba77-81e9e3a71054 None admin] RBAC: Authorizing `identity:get_user(user_id=cd4454f3575c4a22a8aea53ffe9a25c8)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.078063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5bea2-8b23-40fb-ba77-81e9e3a71054 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.079841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 128/266] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/users/cd4454f3575c4a22a8aea53ffe9a25c8 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:52.086454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.242121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800a6eb-2791-4d98-993a-5682de0a3d75 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.285572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.286668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.287177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.287648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.318558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.372372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.374263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.444998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c444dd-6c14-40c8-98ca-568ecc080e91 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.445301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c444dd-6c14-40c8-98ca-568ecc080e91 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.445564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c444dd-6c14-40c8-98ca-568ecc080e91 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.447012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 129/267] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:52.452511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e98c-e66b-4c8e-bab2-98b40b1373f6 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.452791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e98c-e66b-4c8e-bab2-98b40b1373f6 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.453080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e98c-e66b-4c8e-bab2-98b40b1373f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.482285 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-02f7e98c-e66b-4c8e-bab2-98b40b1373f6 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:52.483505 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-02f7e98c-e66b-4c8e-bab2-98b40b1373f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:52.519356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 139/268] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:52.526170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.598963 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.624537 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 130/269] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:52.625898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6084d7-66c5-4dec-93fc-70caebd67421 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.626222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6084d7-66c5-4dec-93fc-70caebd67421 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.626493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6084d7-66c5-4dec-93fc-70caebd67421 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.627787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 131/270] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:52.634336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.634806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.635215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.666269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce338-d004-4033-98bf-1069f4dce509 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.672133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:52.673524 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:52.675972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.711957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.712968 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.715049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.715049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.715049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.716956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.719506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 140/271] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:52.722847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.752870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 132/272] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:52.758546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.854204 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3cf9c-c7ad-4e87-86d1-37a4d9f4d627 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.875052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197969b-697c-470a-b69e-ac392c96c222 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.882638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.895211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26c87d-e193-4057-96b1-c2b9a385396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.896694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26c87d-e193-4057-96b1-c2b9a385396a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.897148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26c87d-e193-4057-96b1-c2b9a385396a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.897918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26c87d-e193-4057-96b1-c2b9a385396a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.909692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.910378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.910660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.910934 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.911244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26c87d-e193-4057-96b1-c2b9a385396a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:52.911803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] RBAC: Adding query filter params (role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=cd4454f3575c4a22a8aea53ffe9a25c8) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:52.912115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.913570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.926328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 133/273] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/role_assignments?user.id=cd4454f3575c4a22a8aea53ffe9a25c8&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2d26c87d-e193-4057-96b1-c2b9a385396a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:52.926511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:52.927901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:52.928687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:52.929284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.929353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 141/274] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:52.929353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0574-e532-401c-ab7a-d7832fd070b6 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.929353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0574-e532-401c-ab7a-d7832fd070b6 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.929353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0574-e532-401c-ab7a-d7832fd070b6 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.929353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 142/275] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:52.938155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.938463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.938463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.938463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.958696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:52.959985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:53.040630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 143/276] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:53.041607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.095657 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6134-eb00-46eb-858d-7f319563785a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.142902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.145316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.145316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.145316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.149296 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.149296 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.151136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 134/277] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.152539 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.178944 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a21b58-4c0d-46e7-973c-c6302cdaab90 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.196218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.216230 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.216992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.217269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.217753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.218321 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:53.218624 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.219982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.229533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 144/278] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.230192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.245175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.278027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f532d94-9003-43a9-86cc-1111a53861db None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.311847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755b169-af53-4420-943f-273a29b193eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.312641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755b169-af53-4420-943f-273a29b193eb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.312926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755b169-af53-4420-943f-273a29b193eb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.313202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755b169-af53-4420-943f-273a29b193eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.324430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755b169-af53-4420-943f-273a29b193eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8755b169-af53-4420-943f-273a29b193eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:53.332254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:53.333148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:53.334570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:53.335159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e36285-4274-4ba2-94b3-dbe2e142ca6c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.335231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 135/279] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:53.335231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755b169-af53-4420-943f-273a29b193eb None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.368203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.368873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.369158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.369433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.370179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.371613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.432460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755b169-af53-4420-943f-273a29b193eb None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.459958 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.465389 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc7360-91b4-4064-94b5-d4f441758bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.466429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc7360-91b4-4064-94b5-d4f441758bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.466713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc7360-91b4-4064-94b5-d4f441758bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.466980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc7360-91b4-4064-94b5-d4f441758bc9 None admin] PATH_INFO: `/v3/users/460a09d31d2946a299e1338d63bd6a4b` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.480236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc7360-91b4-4064-94b5-d4f441758bc9 None admin] RBAC: Authorizing `identity:get_user(user_id=460a09d31d2946a299e1338d63bd6a4b)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.481687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc7360-91b4-4064-94b5-d4f441758bc9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.483352 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 136/280] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/users/460a09d31d2946a299e1338d63bd6a4b => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.484703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3c93a-fe8d-40f3-8ab0-940d0732ccf2 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.484993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3c93a-fe8d-40f3-8ab0-940d0732ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.485260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3c93a-fe8d-40f3-8ab0-940d0732ccf2 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.486508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 137/281] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:53.487189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3c93a-fe8d-40f3-8ab0-940d0732ccf2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.522042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 145/282] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:53 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:53.522042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.556791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.614286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3c93a-fe8d-40f3-8ab0-940d0732ccf2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.650128 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790d49-eacf-41e3-b79b-9c36c34e6f33 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.664427 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.665105 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.665385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.666580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.667467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:53.668305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.669360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220cd4f-f272-4053-8752-86585a769c36 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.682043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 138/283] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.682043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.682276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.686630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.704058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:53.706123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:53.709937 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.710724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.711008 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.711283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.712133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, user.id=460a09d31d2946a299e1338d63bd6a4b, scope.system=all) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:53.712447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.714324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.720111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 146/284] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/role_assignments?user.id=460a09d31d2946a299e1338d63bd6a4b&scope.system=all&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.720943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.791080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.815159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 139/285] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:53.841806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.883262 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.893225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51716523-a635-4d28-b4f5-6f61833410ab None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.969093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.970896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.970896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.971219 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] PATH_INFO: `/v3/users/421b09a23b124ab38ae1c39b23815300` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.973438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=421b09a23b124ab38ae1c39b23815300)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.975195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.977020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 147/286] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/users/421b09a23b124ab38ae1c39b23815300 => generated 297 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.982562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:54.068083 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e03b4-074b-4c71-9fdd-1e303692af8a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:54.157796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:54.160068 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054e7f-36b2-4fff-ae4c-0bfa3a9af6b7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:54.160521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:54.160808 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:54.161079 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:54.185467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:54.203179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:54.204136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:54.204930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:54.207210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 140/287] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:54.213501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:54.249201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:54.251460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:54.253176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:54.253665 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:54.258123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:54.264344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:54.277324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 148/288] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:54.291025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:54.356076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:54.374390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:54.484670 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6e66-cfa8-4fe4-827b-3fe3a21a2136 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:54.535099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2421437-f694-4d72-9ac1-68edf1ff5809 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:54.573236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:54.573963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:54.574267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:54.574540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:54.575307 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:54.575607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:54.576997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:54.599128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 141/289] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:54.601938 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:54.621776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:54.624326 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:54.624326 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:54.624326 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/421b09a23b124ab38ae1c39b23815300/roles/c855898533554ff5b216819dc74c80e8` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:54.626268 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae63c1ef62974692a21343d5415af4fe, user_id=421b09a23b124ab38ae1c39b23815300, role_id=c855898533554ff5b216819dc74c80e8)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:54.627953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:54.741247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0959fb8-6b72-4ed9-a5b7-27b2527df012 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:54.742864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:54.764853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 149/290] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:54 2022] PUT /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe/users/421b09a23b124ab38ae1c39b23815300/roles/c855898533554ff5b216819dc74c80e8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:42:54.790974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:54.791831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:54.792164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:54.792440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] PATH_INFO: `/v3/users/b08ba4e8116144578e013234d3271618` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:54.793748 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] RBAC: Authorizing `identity:get_user(user_id=b08ba4e8116144578e013234d3271618)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:54.795264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:54.797462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 142/291] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:54 2022] GET /identity/v3/users/b08ba4e8116144578e013234d3271618 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:54.804342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:54.940492 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:54.950432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b46bb-01f1-4ac9-8b3a-865cb93569a3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:54.987184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:54.987849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:54.988137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:54.988410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:55.070478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:55.071881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:55.164335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:55.174882 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 150/292] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:55.180907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:55.338084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df302f-7b99-4356-b2f3-b333b4678a87 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:55.392654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba095c0-e786-4744-8d5d-a620a3252401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:55.393533 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba095c0-e786-4744-8d5d-a620a3252401 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:55.393821 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba095c0-e786-4744-8d5d-a620a3252401 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:55.394123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba095c0-e786-4744-8d5d-a620a3252401 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:55.395336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba095c0-e786-4744-8d5d-a620a3252401 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:55.396883 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba095c0-e786-4744-8d5d-a620a3252401 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:55.398758 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 143/293] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:55 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:55.404736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:55.549451 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375336a-165d-4947-ab12-6b515bc78d2b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:55.593714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:55.594325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:55.594620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:55.594882 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:55.595727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=b08ba4e8116144578e013234d3271618) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:55.596038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:55.597482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1f9d9-a38d-4053-9bbd-715810f680d2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:55.604291 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 151/294] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:55 2022] GET /identity/v3/role_assignments?user.id=b08ba4e8116144578e013234d3271618&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:55.992073 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917612fb-4fc6-4b20-bac2-bb2f96b6803b None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:55.992806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917612fb-4fc6-4b20-bac2-bb2f96b6803b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:55.993302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917612fb-4fc6-4b20-bac2-bb2f96b6803b None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:55.995270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 144/295] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:56.000745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.001282 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.001754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.021007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:56.022530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:56.081641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.098113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfc4e8-359c-4022-b583-4441a687d87b None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.098411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfc4e8-359c-4022-b583-4441a687d87b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.098674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfc4e8-359c-4022-b583-4441a687d87b None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.100719 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 145/296] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:56.102803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 152/297] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:56.110166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08903-16a1-405f-a8f4-1ccb088fa8a3 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.110166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08903-16a1-405f-a8f4-1ccb088fa8a3 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.110166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08903-16a1-405f-a8f4-1ccb088fa8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.126757 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.128997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-cef08903-16a1-405f-a8f4-1ccb088fa8a3 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:56.132507 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-cef08903-16a1-405f-a8f4-1ccb088fa8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:56.176650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.231561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 146/298] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:56.239182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704f1c3-d0eb-457a-a674-ee75ac25197b None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.239182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704f1c3-d0eb-457a-a674-ee75ac25197b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.239182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704f1c3-d0eb-457a-a674-ee75ac25197b None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.240860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 147/299] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:56.246028 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.246287 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.246564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.255135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956242a-71a6-45d5-896b-54690dd347a4 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.264659 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:56.265859 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:56.292412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.293240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.293519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.293796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.304971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:56.312692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:56.313626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:56.314495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 153/300] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:56.315171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.341673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 148/301] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:56.341673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.371533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.429007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7086ea-9835-419a-9097-16a6801bc50b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.453422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.457231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d675e-aca7-4359-a679-28c477d82953 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.466025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.466926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.467211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.467490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.478315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:56.486045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:56.486930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:56.487715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 154/302] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:56.488899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.497792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.498442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.498727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.499003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.499764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:56.500068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.501451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.510202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 149/303] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:56.510202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.601412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a3c2-2f50-4f68-bf0c-11b11b032f0a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.604785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.628012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81988b31-d760-4e1b-a1f4-df3b7e9d9fc3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.635645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.636440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.636718 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.636989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.638178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.639669 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.646644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 155/304] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:56.646644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.667769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.668445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.668730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.669008 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.669759 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:56.670090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.671480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.679808 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 150/305] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:56.684319 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.707207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.759195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf338e4-607c-44d6-9ca8-a5a94e4d8239 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.796884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.798546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.798546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.798546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] PATH_INFO: `/v3/users/460a09d31d2946a299e1338d63bd6a4b` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.799589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] RBAC: Authorizing `identity:get_user(user_id=460a09d31d2946a299e1338d63bd6a4b)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.801033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.807576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 156/306] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/users/460a09d31d2946a299e1338d63bd6a4b => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:56.807576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.807862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05529-7507-436a-8973-a38eeada1b13 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.843309 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.843988 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.844274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.844549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] PATH_INFO: `/v3/groups` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.845324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.846801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.854383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.914372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a982-18f6-4616-bca9-950c2ed647f2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.950042 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 151/307] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Jul 25 18:42:56 2022] POST /identity/v3/groups => generated 229 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:56.950042 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.956499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.957529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.957818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.958166 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] PATH_INFO: `/v3/users/6ef96baa8173405e9c76b6a7bc57f9ea` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.975425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] RBAC: Authorizing `identity:get_user(user_id=6ef96baa8173405e9c76b6a7bc57f9ea)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.977113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.979336 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 157/308] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/users/6ef96baa8173405e9c76b6a7bc57f9ea => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:56.985779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.064777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.089522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71e583-731c-4e59-81fb-d16ff5b8da67 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.137012 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.140290 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaae49-1d00-4754-a9c9-4dc072a759e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.144567 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaae49-1d00-4754-a9c9-4dc072a759e0 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.144567 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaae49-1d00-4754-a9c9-4dc072a759e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.144567 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaae49-1d00-4754-a9c9-4dc072a759e0 None admin] PATH_INFO: `/v3/system/users/460a09d31d2946a299e1338d63bd6a4b/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.144838 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aebcfc-108f-4ee2-87d6-ab2b716c71b8 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.144918 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaae49-1d00-4754-a9c9-4dc072a759e0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=460a09d31d2946a299e1338d63bd6a4b, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.147633 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaae49-1d00-4754-a9c9-4dc072a759e0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.185864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.186558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.186861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.187137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.188826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] RBAC: Adding query filter params (role.id=c855898533554ff5b216819dc74c80e8, user.id=6ef96baa8173405e9c76b6a7bc57f9ea, scope.system=all) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:57.189141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.190620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c606e2-6d6a-473d-a931-4672e7a311bb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.197457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 158/309] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/role_assignments?user.id=6ef96baa8173405e9c76b6a7bc57f9ea&scope.system=all&role.id=c855898533554ff5b216819dc74c80e8 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:57.280417 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 152/310] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Jul 25 18:42:56 2022] PUT /identity/v3/system/users/460a09d31d2946a299e1338d63bd6a4b/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:57.315913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12645-0736-4e4e-bf44-04c5c119afac None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.316104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12645-0736-4e4e-bf44-04c5c119afac None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.316385 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12645-0736-4e4e-bf44-04c5c119afac None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.319255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 159/311] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:57.325040 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.330029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.330029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.347056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:57.348735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:57.431969 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 153/312] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:57.455283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12645-0736-4e4e-bf44-04c5c119afac None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.596444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12645-0736-4e4e-bf44-04c5c119afac None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.634589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.635530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.635817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.636145 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.647960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:57.656440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:57.662495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:57.664472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 160/313] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:57.666093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.753556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.777733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45667c98-e692-474d-b9cf-edd4d2278c14 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.820364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.821047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.821332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.821609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.822419 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:57.822726 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.824113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.833036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 154/314] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:57.838903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.866896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.949736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e2dc-7d47-4e94-b452-f772d5f6df33 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.986588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.987519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.987809 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.988086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] PATH_INFO: `/v3/users/421b09a23b124ab38ae1c39b23815300` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.989311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] RBAC: Authorizing `identity:get_user(user_id=421b09a23b124ab38ae1c39b23815300)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.990929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.992585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 161/315] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/users/421b09a23b124ab38ae1c39b23815300 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:57.998421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.108480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d151544-215b-4a8b-be23-7114afd9084e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.142711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.143294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.143627 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.143905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.156864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.216531 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.217999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.320995 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 155/316] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:58.326506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.425811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.435402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aec10-8ea0-4244-8c48-44ec655159a9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.477476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33cd37-126b-47d7-8321-d0f0b9526c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.478383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33cd37-126b-47d7-8321-d0f0b9526c07 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.478668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33cd37-126b-47d7-8321-d0f0b9526c07 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.478948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33cd37-126b-47d7-8321-d0f0b9526c07 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.480162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33cd37-126b-47d7-8321-d0f0b9526c07 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.481604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33cd37-126b-47d7-8321-d0f0b9526c07 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.488438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 162/317] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:58 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:58.489863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.531430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.612431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361bba6-4b0c-4b9e-927c-734ffec79d7f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.648198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.648857 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.649141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.649420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.650476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] RBAC: Adding query filter params (role.id=c855898533554ff5b216819dc74c80e8, scope.project.id=ae63c1ef62974692a21343d5415af4fe, user.id=421b09a23b124ab38ae1c39b23815300) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:58.650789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.652245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39e7d6-c64c-40d0-bc9d-d47a2f8ecae6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.658396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 156/318] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:58 2022] GET /identity/v3/role_assignments?user.id=421b09a23b124ab38ae1c39b23815300&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=c855898533554ff5b216819dc74c80e8 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:59.159818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55a24-43cd-4d8f-b64c-552f870681de None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.160101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55a24-43cd-4d8f-b64c-552f870681de None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.160545 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55a24-43cd-4d8f-b64c-552f870681de None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.162628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 163/319] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:59.167855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef503eca-d4be-40dd-8d03-c8c20a285df1 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.168220 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef503eca-d4be-40dd-8d03-c8c20a285df1 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.168571 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef503eca-d4be-40dd-8d03-c8c20a285df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.187986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ef503eca-d4be-40dd-8d03-c8c20a285df1 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:59.189178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ef503eca-d4be-40dd-8d03-c8c20a285df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:59.266058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 157/320] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:59.286722 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55a24-43cd-4d8f-b64c-552f870681de None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.359344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafe9c0-46a0-48a9-9d4f-8693b5cc73e7 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.359856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafe9c0-46a0-48a9-9d4f-8693b5cc73e7 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.359945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafe9c0-46a0-48a9-9d4f-8693b5cc73e7 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.361242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 158/321] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:59.366416 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138fcf-febb-4ba1-a4ba-3a11427d02da None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.366700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138fcf-febb-4ba1-a4ba-3a11427d02da None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.367147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138fcf-febb-4ba1-a4ba-3a11427d02da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.384887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-1a138fcf-febb-4ba1-a4ba-3a11427d02da None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:59.386458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1a138fcf-febb-4ba1-a4ba-3a11427d02da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:59.400184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55a24-43cd-4d8f-b64c-552f870681de None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:59.414279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:59.434845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd991ed-fa92-4556-9f72-9bb1f10d3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:59.435650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd991ed-fa92-4556-9f72-9bb1f10d3f2e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.435924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd991ed-fa92-4556-9f72-9bb1f10d3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.436196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd991ed-fa92-4556-9f72-9bb1f10d3f2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.437369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd991ed-fa92-4556-9f72-9bb1f10d3f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:59.438830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd991ed-fa92-4556-9f72-9bb1f10d3f2e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:59.440531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 164/322] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:59.441704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e3932-2861-4e83-8611-96d97a29bda5 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.442259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e3932-2861-4e83-8611-96d97a29bda5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.442534 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e3932-2861-4e83-8611-96d97a29bda5 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.443850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 165/323] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:59.446367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e3932-2861-4e83-8611-96d97a29bda5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.458981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 159/324] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:59.460875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.461344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.461765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.474250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:59.482941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:59.484124 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:59.551056 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e3932-2861-4e83-8611-96d97a29bda5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:59.551967 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 160/325] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:59.554905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.587231 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:59.587906 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.588187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.588461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] PATH_INFO: `/v3/groups` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.589155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:59.590696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:59.655608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:59.664662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d77d9-c58b-4b42-969e-871345d022ac None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:59.685604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 166/326] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Jul 25 18:42:59 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:42:59.687911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.698383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:59.699954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.700242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.700520 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.717318 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:59.726343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:59.727325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:42:59.728147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 161/327] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:59.733877 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.793288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:59.809767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ab475-8b31-4412-84c5-cc120ee6e2ac None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:59.852819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:59.853051 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2258c-8639-4db3-be45-8cb32c4de9f6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:59.853136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.853136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.853427 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.865594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:59.875017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:59.876000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:59.876779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:42:59.877364 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 167/328] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:59.883472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.899328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:59.900262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.900540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.900809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.901570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:59.901874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:59.903326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:59.916789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 162/329] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:59.922725 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:59.943164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.011790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fda59-54ad-4093-986d-06534677b064 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.047632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6d30e-7e5b-4f72-8b50-eaa0e0f6cdf0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.062914 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.063935 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.064216 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.064489 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.065250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:00.065553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.067199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.077627 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 168/330] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:00.085457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.097027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.098085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.098367 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.098639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] PATH_INFO: `/v3/users/6ef96baa8173405e9c76b6a7bc57f9ea` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.099902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] RBAC: Authorizing `identity:get_user(user_id=6ef96baa8173405e9c76b6a7bc57f9ea)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.101357 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.101493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.107831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 163/331] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:59 2022] GET /identity/v3/users/6ef96baa8173405e9c76b6a7bc57f9ea => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:00.110258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.222536 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.233195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853af4-20bc-4dc8-958e-98de70adee91 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.234577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801661c4-f704-4a0a-bade-f22afc2e1362 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.277668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763bcc5-45a8-4865-a667-18eddfb1f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.278587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bcc5-45a8-4865-a667-18eddfb1f77d None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.278872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bcc5-45a8-4865-a667-18eddfb1f77d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.279141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bcc5-45a8-4865-a667-18eddfb1f77d None admin] PATH_INFO: `/v3/system/users/6ef96baa8173405e9c76b6a7bc57f9ea/roles/c855898533554ff5b216819dc74c80e8` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.280853 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763bcc5-45a8-4865-a667-18eddfb1f77d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ef96baa8173405e9c76b6a7bc57f9ea, role_id=c855898533554ff5b216819dc74c80e8)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.280971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.281832 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.282143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.282420 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] PATH_INFO: `/v3/users/460a09d31d2946a299e1338d63bd6a4b` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.283045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763bcc5-45a8-4865-a667-18eddfb1f77d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.283672 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] RBAC: Authorizing `identity:get_user(user_id=460a09d31d2946a299e1338d63bd6a4b)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.285340 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.287756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 169/332] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/users/460a09d31d2946a299e1338d63bd6a4b => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:00.293751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.371612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 164/333] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Jul 25 18:43:00 2022] PUT /identity/v3/system/users/6ef96baa8173405e9c76b6a7bc57f9ea/roles/c855898533554ff5b216819dc74c80e8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:00.382047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.410879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea9bc-dc54-44ea-bb79-4bfbfee63de0 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.453097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.453831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.454165 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.454444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.455303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] RBAC: Adding query filter params (role.id=e587a7d6bedf4b6dabf88e847607ef97, user.id=460a09d31d2946a299e1338d63bd6a4b, scope.system=all) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:00.455614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.457106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d04ef-e902-4f40-95ac-1d096e8324ad None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.472143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 170/334] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/role_assignments?user.id=460a09d31d2946a299e1338d63bd6a4b&scope.system=all&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:02.206625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4181d11-d0c5-4ec5-a901-8533652e90e9 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.207663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4181d11-d0c5-4ec5-a901-8533652e90e9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.208138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4181d11-d0c5-4ec5-a901-8533652e90e9 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.210055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 165/335] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:02.216444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.217304 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.217849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.239664 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:02.241942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:02.338553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 171/336] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:02.367395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4181d11-d0c5-4ec5-a901-8533652e90e9 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.479524 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4181d11-d0c5-4ec5-a901-8533652e90e9 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.517423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e8498-5444-40a2-9f2d-5756e71b1ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.518711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8498-5444-40a2-9f2d-5756e71b1ac1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.519189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8498-5444-40a2-9f2d-5756e71b1ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.519667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8498-5444-40a2-9f2d-5756e71b1ac1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.532046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e8498-5444-40a2-9f2d-5756e71b1ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b97e8498-5444-40a2-9f2d-5756e71b1ac1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:02.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:02.553018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:02.554790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:02.557281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 166/337] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:02.563312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.653547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.706770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e6ba4-0264-462f-927a-e1c6603dc5a0 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.740585 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe9ad9-bc92-4ddf-8225-d43e58725088 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.741758 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe9ad9-bc92-4ddf-8225-d43e58725088 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.742251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe9ad9-bc92-4ddf-8225-d43e58725088 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.743840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 167/338] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:02.749235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac585f-f506-47d9-bf97-11f6ee98e562 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.749784 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac585f-f506-47d9-bf97-11f6ee98e562 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.750277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac585f-f506-47d9-bf97-11f6ee98e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.751067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.751763 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.752049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.752325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.753102 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:02.753450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.754887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.770951 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.777926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 172/339] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:02.777926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.789915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-59ac585f-f506-47d9-bf97-11f6ee98e562 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:02.789915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-59ac585f-f506-47d9-bf97-11f6ee98e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:02.998465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 168/340] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:02.999796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdf921-db6f-44ce-8128-f75bdeb9e07e None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.000086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdf921-db6f-44ce-8128-f75bdeb9e07e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.000370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdf921-db6f-44ce-8128-f75bdeb9e07e None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.006576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 169/341] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:03.007765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.008054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.012043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.052217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:03.053995 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:03.064847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5fd3-c377-46e7-bf2c-08cfe2dc1789 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.117953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.133013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d871a-e91b-47b5-8a63-847a6f66a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.133678 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d871a-e91b-47b5-8a63-847a6f66a559 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.134608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d871a-e91b-47b5-8a63-847a6f66a559 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.134886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d871a-e91b-47b5-8a63-847a6f66a559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.208023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 170/342] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:03.208962 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.278621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d871a-e91b-47b5-8a63-847a6f66a559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.280088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d871a-e91b-47b5-8a63-847a6f66a559 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.290719 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.351445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846a8fe-3132-48a3-beaa-d81289597a26 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.396848 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.398238 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.398519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.398788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.413515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:03.422474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:03.423415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:03.424200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 171/343] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:03.430581 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.464344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 173/344] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:43:03.465795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ffdfe-13a6-41e2-b064-674247566110 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.466109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ffdfe-13a6-41e2-b064-674247566110 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.466387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ffdfe-13a6-41e2-b064-674247566110 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.467681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 174/345] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:03.468374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ffdfe-13a6-41e2-b064-674247566110 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.503933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.575997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327dc56-67e7-4d7f-9398-5bf9ac0c3c21 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.606396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.607137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ffdfe-13a6-41e2-b064-674247566110 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.616400 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dd60e-2c8c-4c7a-820f-9cf36058750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.617497 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dd60e-2c8c-4c7a-820f-9cf36058750d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.622231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dd60e-2c8c-4c7a-820f-9cf36058750d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.622231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dd60e-2c8c-4c7a-820f-9cf36058750d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.632426 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dd60e-2c8c-4c7a-820f-9cf36058750d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-aa6dd60e-2c8c-4c7a-820f-9cf36058750d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:03.641548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:03.642700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:03.643754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:03.647018 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 172/346] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:03.649088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.649572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.650082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.653805 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.654561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.654854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.655132 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.655900 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.656200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.658094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.669353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 175/347] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.674258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:03.674258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:03.675509 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.758158 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 173/348] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:03.758891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.824421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.840211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572325d-0225-414c-89ed-6335329e7a1c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.879344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.880310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.880660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.881004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.883613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.885180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.887497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 176/349] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.888392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.927111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38d2e6-889c-4768-a41d-5e9f34ab7c38 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.951378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.973234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.974445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.975511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.975935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.977113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.977579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.980259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.992438 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 174/350] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.993792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.017930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec8e8b-f1ad-45d9-980a-575e2f1faa7d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.054189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.054962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.055125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.055404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.088281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.144919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.147345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.147544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513d126-3a66-45bf-8f81-7915f2bb87bb None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.212842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.213662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.214400 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.214685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.233531 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:04.251668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:04.252618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:43:04.253408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:04.254030 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 175/351] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:04.255081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.277324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 177/352] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:04.282981 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.331527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.356898 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.418155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394cdd9-55dc-40cf-b2bc-d6c7e784f7bb None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.420188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee936-c882-4bb8-a9c5-4163fe0a574c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.459141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa81eb6-74b4-4de8-b9cf-d6c85952f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.460269 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa81eb6-74b4-4de8-b9cf-d6c85952f090 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.460726 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa81eb6-74b4-4de8-b9cf-d6c85952f090 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.461145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa81eb6-74b4-4de8-b9cf-d6c85952f090 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.467126 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b636484-6c0f-42ba-a132-b0de1b40f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.468199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636484-6c0f-42ba-a132-b0de1b40f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.468734 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636484-6c0f-42ba-a132-b0de1b40f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.469778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636484-6c0f-42ba-a132-b0de1b40f0e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.474888 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa81eb6-74b4-4de8-b9cf-d6c85952f090 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.476916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa81eb6-74b4-4de8-b9cf-d6c85952f090 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.479025 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 176/353] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.480743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a405bf5-4d5e-41fb-8211-61752b2a21b2 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.480743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a405bf5-4d5e-41fb-8211-61752b2a21b2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.480951 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a405bf5-4d5e-41fb-8211-61752b2a21b2 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.482723 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 177/354] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:04.483090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a405bf5-4d5e-41fb-8211-61752b2a21b2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.561513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b636484-6c0f-42ba-a132-b0de1b40f0e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.565272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b636484-6c0f-42ba-a132-b0de1b40f0e0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.624823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a405bf5-4d5e-41fb-8211-61752b2a21b2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.625974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.637275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.665281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.666421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.666421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.666750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.667460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:04.667763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.669203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.680707 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 178/355] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.681621 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.689605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 178/356] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:04.690812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee570a-fc84-406b-b818-c756740fd960 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.691111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee570a-fc84-406b-b818-c756740fd960 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.691395 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee570a-fc84-406b-b818-c756740fd960 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.692687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 179/357] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:04.698755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee570a-fc84-406b-b818-c756740fd960 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.804092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61009fcf-d3f2-4713-b085-6a9902ad968a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.831481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee570a-fc84-406b-b818-c756740fd960 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.845089 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9e8bb-32e8-470c-abd1-dcc3d76aff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.846076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9e8bb-32e8-470c-abd1-dcc3d76aff6e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.846361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9e8bb-32e8-470c-abd1-dcc3d76aff6e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.846631 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9e8bb-32e8-470c-abd1-dcc3d76aff6e None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.847828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9e8bb-32e8-470c-abd1-dcc3d76aff6e None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.849265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9e8bb-32e8-470c-abd1-dcc3d76aff6e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.851817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 179/358] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.852800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.853087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.853382 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.876710 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:04.877419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.877855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:04.878907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.878907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.878907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.879751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=3cbb86611f7b4363a6d822a20268eb19) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:04.880259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.881979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.888029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 180/359] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.888621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.917682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.924965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.977135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 180/360] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:04.978340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.048604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206c62e-12a6-4072-acbf-50b01c235a53 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.099819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326eeb4-093d-45a6-9fc4-81fb96495710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.101215 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326eeb4-093d-45a6-9fc4-81fb96495710 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.101499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326eeb4-093d-45a6-9fc4-81fb96495710 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.101773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326eeb4-093d-45a6-9fc4-81fb96495710 None admin] PATH_INFO: `/v3/users/6ef96baa8173405e9c76b6a7bc57f9ea` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.103684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326eeb4-093d-45a6-9fc4-81fb96495710 None admin] RBAC: Authorizing `identity:get_user(user_id=6ef96baa8173405e9c76b6a7bc57f9ea)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.105262 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326eeb4-093d-45a6-9fc4-81fb96495710 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.107359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 181/361] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/users/6ef96baa8173405e9c76b6a7bc57f9ea => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.108739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.109027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.109323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.111577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b5582-4d28-425b-87f9-0dd200deb79a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.128317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:05.129435 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:05.151595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.152913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.153500 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.153985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.155564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.157337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.159805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 181/362] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.160800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.198047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.203411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 182/363] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:05.204751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.227685 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.291915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d8387-1ec2-452c-a57c-0da37e3e04dd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.332432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.333552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.333979 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.334381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83c86f-17e7-4db8-9c1b-864ef73209a6 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.334895 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.336276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.338048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.339732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 182/364] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.340516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.380054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.380817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.381103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.381379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.394063 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:05.405593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:05.407363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 183/365] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:05.409575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.467060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.471527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6249f44-7493-4376-9a2a-e457ba8360bd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.517716 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.518411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.518693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.518966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.519804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] RBAC: Adding query filter params (role.id=c855898533554ff5b216819dc74c80e8, user.id=6ef96baa8173405e9c76b6a7bc57f9ea, scope.system=all) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.520126 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.521903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.542146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 183/366] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/role_assignments?user.id=6ef96baa8173405e9c76b6a7bc57f9ea&scope.system=all&role.id=c855898533554ff5b216819dc74c80e8 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.543192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.544330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.565404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f64f92-7520-4ae5-8e85-c7fa4461dc90 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.618952 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.620094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.620557 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.620981 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.623190 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.625427 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.628884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 184/367] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.632955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.703141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b3afa-f447-4259-94a8-5e01e9dc6c4b None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.745572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.746552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.746847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.747122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.765077 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:05.775916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:05.776841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:05.777763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:05.778521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794627a-7f8f-4856-8039-c18b8f7ab079 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.778606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 184/368] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:05.778606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.801175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.824749 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.825660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.826160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.826585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.828573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=3cbb86611f7b4363a6d822a20268eb19) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.828897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.830392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.836557 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 185/369] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.838583 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.855085 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.925641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe65077-8678-4635-b799-67cc54130865 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.978740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.980003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.980438 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.980857 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.982351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.982940 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.984681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.986301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6175-382a-4b73-99fa-ad6886d09000 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.002680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 185/370] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.006169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.044618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.045858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.046389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.046873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.048014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=ae63c1ef62974692a21343d5415af4fe) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:06.048524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.050330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.057382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 186/371] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.058713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.098145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.121529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19777512-eaed-4650-89f9-ebf95662e8d6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.151063 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.190603 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.191304 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.191826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.192107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.193263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:06.193578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.201213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.214566 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee155fd0-ce1f-4f5a-87a6-48c479ee3697 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.224321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 186/372] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.230637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.280352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.281017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.281301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.281874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.358770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.362349 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.408511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348db026-b2b5-4411-b4d7-042dc108422f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.458439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.459433 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.460471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.460966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.465761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.479267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.480690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 187/373] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:06.488523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.608479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.609970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.666883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d778b-bbb5-4052-8f71-714d132600d9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.725079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.726314 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.726799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.727304 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.728891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.730633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.733102 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 188/374] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.738167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.755104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 187/375] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:43:06.761140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.798035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.820024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.902444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f2547-f130-49c4-a75c-dbc14643fa84 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.932997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbc2b7-a169-4fd8-8b72-0b4ae803ba09 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.941569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.942786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.944987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.945448 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.951480 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.953127 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.955209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 189/376] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.962977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.011298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.014168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.015829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.016420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] PATH_INFO: `/v3/projects/16218857a48d4501b2b8db4e0e5b32f7` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.018752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] RBAC: Authorizing `identity:get_project(project_id=16218857a48d4501b2b8db4e0e5b32f7)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.021916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.024271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 188/377] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/projects/16218857a48d4501b2b8db4e0e5b32f7 => generated 344 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.031008 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.107897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbef7c6-c2db-4dab-8eef-344be6ab0d96 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.130395 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.150049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.150049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.150049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.150049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.153624 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=ae63c1ef62974692a21343d5415af4fe) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:07.153624 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.153624 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.160289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 190/378] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.247143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f895c-99a2-4dc2-ac86-d1d6fb311d1c None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.264248 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.299656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85815201-8f41-4361-b532-a64779e7b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.300523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85815201-8f41-4361-b532-a64779e7b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.300799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85815201-8f41-4361-b532-a64779e7b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.301066 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85815201-8f41-4361-b532-a64779e7b2d7 None admin] PATH_INFO: `/v3/groups/563d7d219a3e4b83a4d7fd6cbc5085b5` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.310824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85815201-8f41-4361-b532-a64779e7b2d7 None admin] RBAC: Authorizing `identity:get_group(group_id=563d7d219a3e4b83a4d7fd6cbc5085b5)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.312328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85815201-8f41-4361-b532-a64779e7b2d7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.314137 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 189/379] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/groups/563d7d219a3e4b83a4d7fd6cbc5085b5 => generated 229 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.320676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.339467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d773e0c-b218-491c-b635-874027fc3a7d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.339772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d773e0c-b218-491c-b635-874027fc3a7d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.340210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d773e0c-b218-491c-b635-874027fc3a7d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.342416 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 190/380] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:07.356569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.356874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.357129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.387687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:07.390561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:07.483367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067a8e7-2591-47a4-b47e-abd2fb4e2db5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.519816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 191/381] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:07.532797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.533545 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.533908 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.534196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.535102 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] RBAC: Adding query filter params (group.id=563d7d219a3e4b83a4d7fd6cbc5085b5, role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=16218857a48d4501b2b8db4e0e5b32f7) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:07.535414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.536894 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.550541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 191/382] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/role_assignments?group.id=563d7d219a3e4b83a4d7fd6cbc5085b5&scope.project.id=16218857a48d4501b2b8db4e0e5b32f7&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.580378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.766833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42d66c-4a0a-4b8e-ad42-263812ecf29c None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.770406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.825828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.826700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.826981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.827253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.842655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:07.851517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:07.852397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:07.853208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:07.856013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 192/383] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:07.863338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.933726 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.023695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06fbbd-48b6-4407-97ca-ec29878c8b32 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.072977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.073655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.073937 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.074241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.074992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:08.075295 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.076669 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f3626-22d9-420d-a4ff-3d387134bf1c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.090157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 192/384] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:08.095820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.187805 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671f3b6-ba01-46ea-b29d-5f1111941821 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.188103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671f3b6-ba01-46ea-b29d-5f1111941821 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.188376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671f3b6-ba01-46ea-b29d-5f1111941821 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.189788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 193/385] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:08.204464 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.204747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.205027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.239183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:08.240623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:08.278112 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf66dc8-4bdb-4c87-ad84-83c9651e1abf None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.345029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.345906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.346214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.346497 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.349042 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.350574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.354173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 193/386] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:08.358226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.380223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 194/387] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:08.403059 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.425670 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.436043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.558729 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27198b-979a-4b05-9214-b37873cb2b72 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.577460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d25da-8c24-440c-adba-6ed57a9306ae None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.623847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.624737 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.625020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.625294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.629777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.631272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.633468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 194/388] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:08.641899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.654863 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.655678 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.655962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.656231 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.677232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:08.685360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:08.686307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:08.687107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:08.697733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 195/389] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:08.705020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.800580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.806059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f298a-494d-48ec-93d9-58d284228f32 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.820680 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.830720 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea10361-7162-4ac4-a349-e3e3cef67ebe None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.844330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455582a-6cad-480a-aa2f-01a71182df0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.845521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455582a-6cad-480a-aa2f-01a71182df0f None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.846032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455582a-6cad-480a-aa2f-01a71182df0f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.846511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455582a-6cad-480a-aa2f-01a71182df0f None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19/groups/e160d641702244ebbea310aed5aa4a89/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.849049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455582a-6cad-480a-aa2f-01a71182df0f None admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb86611f7b4363a6d822a20268eb19, group_id=e160d641702244ebbea310aed5aa4a89, role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.850783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455582a-6cad-480a-aa2f-01a71182df0f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.867828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.868497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.871704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.871704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.871704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:08.871704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.871704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535dfcc-7bc0-4d81-84b7-5c0e6c384b23 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.880416 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 196/390] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:08.885693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95e05-63bd-404d-a724-1b2445771cfa None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.885693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95e05-63bd-404d-a724-1b2445771cfa None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.885693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95e05-63bd-404d-a724-1b2445771cfa None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.885693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 197/391] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:08.886439 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95e05-63bd-404d-a724-1b2445771cfa None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.933363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 195/392] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 25 18:43:08 2022] PUT /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19/groups/e160d641702244ebbea310aed5aa4a89/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:08.936410 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.941505 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.941972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.992328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:08.993594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:09.086123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95e05-63bd-404d-a724-1b2445771cfa None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.137661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 196/393] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:09.147577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.148982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.149257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.149522 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.151827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.153331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.154993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 198/394] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:09.160734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.175841 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.199170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.271646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.285157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe02a83-bc4f-41df-826f-0d17ddc72b73 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.296880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eb49d-71af-4446-9534-55aa50a0ba63 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.321483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.322440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.322729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.323006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.324212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.325681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.327735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 197/395] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:09.333246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.343928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.343928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.343928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.343928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.356280 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:09.364437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:09.366999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:09.368982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 199/396] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:09.378034 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.458927 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.479771 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb487fe9-5c23-42f5-9e22-564642c2439f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.516636 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.517308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.517587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.517860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.518700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:09.519004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.520415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.529980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 200/397] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:09.535581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.617050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ec11-ad6b-4e01-9988-8b065c61e3e5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.643870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6fe0e-8106-4785-8953-3da609750070 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.681050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.681916 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.682264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.682550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.683782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.685315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.691784 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 201/398] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:09.692973 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.707739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.716952 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a42257-0bcf-4d7f-915d-1b48561ceabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.718068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a42257-0bcf-4d7f-915d-1b48561ceabf None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.718506 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a42257-0bcf-4d7f-915d-1b48561ceabf None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.718884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a42257-0bcf-4d7f-915d-1b48561ceabf None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19/groups/e160d641702244ebbea310aed5aa4a89/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.728068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a42257-0bcf-4d7f-915d-1b48561ceabf None admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb86611f7b4363a6d822a20268eb19, group_id=e160d641702244ebbea310aed5aa4a89, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.729761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a42257-0bcf-4d7f-915d-1b48561ceabf None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.829181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07caf5-8e5f-44f0-a688-4b1a0b05369d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.834397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 198/399] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 25 18:43:09 2022] PUT /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19/groups/e160d641702244ebbea310aed5aa4a89/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:43:09.836824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec2355-d071-4305-9820-8b9c2452779a None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.837286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec2355-d071-4305-9820-8b9c2452779a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.837725 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec2355-d071-4305-9820-8b9c2452779a None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.840774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 199/400] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:09.852141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc984848-176d-481d-ab4b-161bbc79e1b2 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.852713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc984848-176d-481d-ab4b-161bbc79e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.853168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc984848-176d-481d-ab4b-161bbc79e1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.891365 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-dc984848-176d-481d-ab4b-161bbc79e1b2 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:09.894460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-dc984848-176d-481d-ab4b-161bbc79e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:09.915797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.916983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.917452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.918366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.921101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.922413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.924332 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.933735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 202/401] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:09.939822 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.017261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 200/402] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:10.018952 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f9682-6af3-464c-bc05-d47c543f0883 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.019650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f9682-6af3-464c-bc05-d47c543f0883 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.019922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f9682-6af3-464c-bc05-d47c543f0883 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.021729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 201/403] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:10.031230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.031537 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.031772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.054809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:10.055998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:10.057735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.093535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c18acb-8ce7-48f7-af94-c57aa0110cdb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.145431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 202/404] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:10.146245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.147584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.149185 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.149472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.149749 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe/groups/e160d641702244ebbea310aed5aa4a89/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.151836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] RBAC: Authorizing `identity:create_grant(project_id=ae63c1ef62974692a21343d5415af4fe, group_id=e160d641702244ebbea310aed5aa4a89, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.153405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.244041 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.271402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60cf87-a95d-43b0-927a-ac8f4412920a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.279069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 203/405] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 25 18:43:09 2022] PUT /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe/groups/e160d641702244ebbea310aed5aa4a89/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:43:10.279069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.352717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.353620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.353904 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.354250 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.379639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:10.392588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.393553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:10.394376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:10.395724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 203/406] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.403706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.487123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb8962-1831-48cf-b6a0-b5ed6fb1d99a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.525491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.526618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.526618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.526926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.534056 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.539548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:10.547411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.558811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:10.560488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 204/407] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.564320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.581481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548bc53-92e3-430b-9fdb-339d50ec6898 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.626717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.628244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.628747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.629660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.630948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:10.631459 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.634351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.641672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.645996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 204/408] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:10.653399 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.705324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474604a-fbb9-4f5c-8f0e-23590e8c1106 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.770392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.771070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.771356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.771634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.772402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:10.772708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.774150 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.797992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 205/409] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:10.798966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.847635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76cc8f-3ce8-411b-be51-14c97e8b9057 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.906460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.909355 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.910260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.910544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.910819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.912138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.913586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.923687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 205/410] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:10.924616 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.986816 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a18e8-60f9-4e27-9eb3-a7a3afbe87d3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.045091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.055634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.056518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.056803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.057234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] PATH_INFO: `/v3/groups/563d7d219a3e4b83a4d7fd6cbc5085b5` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.063713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] RBAC: Authorizing `identity:get_group(group_id=563d7d219a3e4b83a4d7fd6cbc5085b5)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.065194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.067174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 206/411] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/groups/563d7d219a3e4b83a4d7fd6cbc5085b5 => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.074661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.122799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86183a-6f37-45b1-a747-bc26406b84d4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.190401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.191184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.191470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.191745 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.193740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.195527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.202427 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 206/412] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.207650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.256617 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c8e61-ec08-4e39-8bf1-cd760d34dc79 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.285457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.294716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b276-c549-4aa7-a8c6-62b067f43acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.295585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b276-c549-4aa7-a8c6-62b067f43acb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.295906 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b276-c549-4aa7-a8c6-62b067f43acb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.296181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b276-c549-4aa7-a8c6-62b067f43acb None admin] PATH_INFO: `/v3/projects/16218857a48d4501b2b8db4e0e5b32f7` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.297659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235b276-c549-4aa7-a8c6-62b067f43acb None admin] RBAC: Authorizing `identity:get_project(project_id=16218857a48d4501b2b8db4e0e5b32f7)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.299149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235b276-c549-4aa7-a8c6-62b067f43acb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.303104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 207/413] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/projects/16218857a48d4501b2b8db4e0e5b32f7 => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:11.303104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9155a7-e6d6-4c22-a65a-abfd96602f8d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.303104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9155a7-e6d6-4c22-a65a-abfd96602f8d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.303104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9155a7-e6d6-4c22-a65a-abfd96602f8d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.304384 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 208/414] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:11.311035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.311330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.311612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.332348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:11.333665 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:11.342443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097ed95-9a57-4411-a7fa-ee5a851443d2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.377116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.378003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.378310 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.378588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe/groups/e160d641702244ebbea310aed5aa4a89/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.380829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae63c1ef62974692a21343d5415af4fe, group_id=e160d641702244ebbea310aed5aa4a89, role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.382317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.384760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.407813 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 209/415] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:11.408847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.494501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 207/416] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 25 18:43:11 2022] PUT /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe/groups/e160d641702244ebbea310aed5aa4a89/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:11.495474 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.516614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a459cf-7345-4cbb-9c12-85f9e85a3db0 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.557313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.576272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.587804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.587804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.587804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] PATH_INFO: `/v3/projects/16218857a48d4501b2b8db4e0e5b32f7/groups/563d7d219a3e4b83a4d7fd6cbc5085b5/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.587804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] RBAC: Authorizing `identity:create_grant(project_id=16218857a48d4501b2b8db4e0e5b32f7, group_id=563d7d219a3e4b83a4d7fd6cbc5085b5, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.587804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.654700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85106c-2f04-4cc7-9a32-1fbda9927e13 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.696057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.697236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.697668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.698113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.708645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 210/417] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 25 18:43:11 2022] PUT /identity/v3/projects/16218857a48d4501b2b8db4e0e5b32f7/groups/563d7d219a3e4b83a4d7fd6cbc5085b5/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:11.713037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:11.722467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:11.723691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:11.724679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:11.728166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 208/418] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:11.734024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.886324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8eeaf-3619-482c-9d08-49ecce4f9c03 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.930561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.931242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.931526 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.931803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.932559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:11.932864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.934260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a603-a422-4acb-9959-5261a1e623e9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.943239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 211/419] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.949735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.014108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.062842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f3d53-387c-406a-8c41-517a744da865 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.098412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.099076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.099364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.099643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.181300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.182733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.191447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f666d-b2f6-4b52-af23-700ffa341c0f None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.191755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f666d-b2f6-4b52-af23-700ffa341c0f None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.192006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f666d-b2f6-4b52-af23-700ffa341c0f None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.193631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 212/420] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:12.211535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.211826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.212105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.246354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:12.247572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:12.254449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.315478 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 209/421] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:12.321696 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.337913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.387687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 213/422] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:12.409950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.515557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc61c5-2269-4a55-8cdc-4c15ae129bb8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.562676 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.563555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.563834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.564103 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.565358 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.566837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.568969 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 210/423] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.576378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.602506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fed9a-bb2a-45ad-b14b-42763cc5c4df None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.641158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.641974 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.642284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.642559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.644939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.656034 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:12.665039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:12.665942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:12.666775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 214/424] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:12.672924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.718450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.788195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b6a9c-3017-4e28-80e3-2d456cad12fb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.823511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcf5a-73b4-426e-984e-c94e1e442049 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.844031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.844706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.844991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.845266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.846035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] RBAC: Adding query filter params (name=member) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:12.846348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.847852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1fcf9-396b-4516-8a67-6a9457056e61 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.869925 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 215/425] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.875381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623ff5c-d63f-44af-a0bb-6725d1054ec8 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.875674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623ff5c-d63f-44af-a0bb-6725d1054ec8 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.875956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623ff5c-d63f-44af-a0bb-6725d1054ec8 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.877231 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 216/426] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:12.878544 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623ff5c-d63f-44af-a0bb-6725d1054ec8 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.888866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ebc18-b3d2-4c1f-8e54-2871210818fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.890873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ebc18-b3d2-4c1f-8e54-2871210818fb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.891305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ebc18-b3d2-4c1f-8e54-2871210818fb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.891712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ebc18-b3d2-4c1f-8e54-2871210818fb None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.894667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ebc18-b3d2-4c1f-8e54-2871210818fb None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.898117 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ebc18-b3d2-4c1f-8e54-2871210818fb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.902302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 211/427] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.904118 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.904536 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.905495 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.940146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:12.941386 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:12.993348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.997796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623ff5c-d63f-44af-a0bb-6725d1054ec8 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.034077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.035687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.035993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.036275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.050200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 212/428] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:13.051033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.086131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.118527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.119975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.167198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76344459-f4dc-4d72-b777-e261baaae026 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.204519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.205184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.205462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.205792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.206678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=3cbb86611f7b4363a6d822a20268eb19) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.206989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.208416 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.215485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 213/429] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:13.216418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.231561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.241199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 217/430] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:43:13.247530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.313833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.339595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ff6a3-d4f2-4ca8-ac16-15c0ec6df348 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.389987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.390817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.391096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.391358 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.405404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:13.414527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:13.416020 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:13.416841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 214/431] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:13.422552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.460348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f96094-d9b5-4973-8cd2-edde3ca4dee2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.531256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.533538 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.533823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.534274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] PATH_INFO: `/v3/projects/3cbb86611f7b4363a6d822a20268eb19` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.535526 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] RBAC: Authorizing `identity:get_project(project_id=3cbb86611f7b4363a6d822a20268eb19)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.536975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.537088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040fd7e-718c-4dff-84c7-41f60eb006dc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.539054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 218/432] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/projects/3cbb86611f7b4363a6d822a20268eb19 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:13.556141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.582913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.583776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.584072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.584347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.585114 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] RBAC: Adding query filter params (name=member) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.585569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.587452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.603902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 215/433] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:13.605948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.620864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.646414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.688945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821603-e99d-4da1-9fb1-a8cb749c501b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.732399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.734072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.734582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.735238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.736779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.738651 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.745211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625ef8-e8f9-4360-aa12-2e9975b5cd82 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.745791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 219/434] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:13.748106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.784686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.785342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.785627 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.785906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.859792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6345c-8fe7-436c-8a6f-0d90883ced03 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.860350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.861819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.870774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.900430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.901495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.902003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.902516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.903626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=3cbb86611f7b4363a6d822a20268eb19) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.904137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.905830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e84f3-25d8-4930-a9fd-b0a5354b3481 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.909811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.922208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 220/435] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=3cbb86611f7b4363a6d822a20268eb19&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:13.970616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9c7b-7f36-4c5e-91e3-f47c0da17c2a None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.970901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9c7b-7f36-4c5e-91e3-f47c0da17c2a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.971188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9c7b-7f36-4c5e-91e3-f47c0da17c2a None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.972919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 221/436] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:13.978504 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 216/437] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:13.978685 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.978941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.979221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.984006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.001713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:14.003118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:14.102289 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f403455-67c0-4ddd-90ce-c31c8bae7dcf None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.106236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 222/438] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:14.111070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.129465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.143576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf5235-2fa2-4de1-97bf-b61cfd4f1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.144454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf5235-2fa2-4de1-97bf-b61cfd4f1d05 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.144730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf5235-2fa2-4de1-97bf-b61cfd4f1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.145002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf5235-2fa2-4de1-97bf-b61cfd4f1d05 None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.147678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf5235-2fa2-4de1-97bf-b61cfd4f1d05 None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.151451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf5235-2fa2-4de1-97bf-b61cfd4f1d05 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.158052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 217/439] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:14.158052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa1ff-86eb-4b14-8db2-ac3a14e1d0b5 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.158052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa1ff-86eb-4b14-8db2-ac3a14e1d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.158052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa1ff-86eb-4b14-8db2-ac3a14e1d0b5 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.158762 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 218/440] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:14.162941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daa1ff-86eb-4b14-8db2-ac3a14e1d0b5 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.259442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6893-8531-441e-a4bc-2083be7ddfb9 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.291192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daa1ff-86eb-4b14-8db2-ac3a14e1d0b5 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.298879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.310733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1465-5b7e-41f7-b9f0-5e7e0cd0c0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.311867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1465-5b7e-41f7-b9f0-5e7e0cd0c0f3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.312344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1465-5b7e-41f7-b9f0-5e7e0cd0c0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.312763 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1465-5b7e-41f7-b9f0-5e7e0cd0c0f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.327258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a1465-5b7e-41f7-b9f0-5e7e0cd0c0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:14.334125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.335236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.335524 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.335800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-542a1465-5b7e-41f7-b9f0-5e7e0cd0c0f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:14.337200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:14.338313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:14.339227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:14.339832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.339902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 223/441] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:14.339968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.340043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.340467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.340623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.343361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 219/442] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:14.345593 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.361703 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:14.363792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:14.395504 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.447172 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 224/443] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:14.447922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.457583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3101ebd-56a5-467c-bcba-ea06e199e8e2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.497819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.498547 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.498824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.499098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.499857 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:14.500160 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.501543 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.510351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 220/444] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:14.511200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.558505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768fc7-0d0e-4d81-909e-a00e35a9e607 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.579228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.594148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.594818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.595120 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.595393 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.596235 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=e587a7d6bedf4b6dabf88e847607ef97, scope.project.id=ae63c1ef62974692a21343d5415af4fe) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:14.596548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.598039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.603999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 225/445] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=e587a7d6bedf4b6dabf88e847607ef97 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:14.604783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.624896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c06f9-c933-463e-b8c1-533bd8a2c88a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.626839 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.664530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.665357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.665643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.665925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.681864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:14.691388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:14.692385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:14.693165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 221/446] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:14.701165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.734364 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd15fff-ebd6-48d5-8663-1f9254d3cd6e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.777653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.778456 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.778829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.779187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.832569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743fd0-7256-46ed-a045-524cffd393ef None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.843552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.864552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.866635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.879866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.880477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.880763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.881042 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.881813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:14.882917 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.884429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.895501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 222/447] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:14.901336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.936573 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.000051 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 226/448] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:15.006034 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.032470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe0b21-9b32-48f2-8c37-442f8929bd82 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.073621 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.074499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.074896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.075261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.121701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.139849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d23fa6-2a24-4d62-acbb-bd6a45ef6cde None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.152632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.154128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.175666 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.176532 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.176807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.177078 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] PATH_INFO: `/v3/projects/ae63c1ef62974692a21343d5415af4fe` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.178275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] RBAC: Authorizing `identity:get_project(project_id=ae63c1ef62974692a21343d5415af4fe)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.179686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.181732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.181937 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 227/449] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/projects/ae63c1ef62974692a21343d5415af4fe => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:15.187436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.253404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 223/450] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:15.258983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.285459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d4ad-5e19-4c5c-bf55-cf6122a6b30f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.320352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.321236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.321510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.321778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] PATH_INFO: `/v3/groups/e160d641702244ebbea310aed5aa4a89` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.322977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] RBAC: Authorizing `identity:get_group(group_id=e160d641702244ebbea310aed5aa4a89)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.324418 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.326050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 228/451] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/groups/e160d641702244ebbea310aed5aa4a89 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:15.331888 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.353429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.363363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e1f4-296a-4e6a-806e-209bff6bcafa None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.389953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.398399 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.399358 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.399638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.399912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] PATH_INFO: `/v3/projects/16218857a48d4501b2b8db4e0e5b32f7` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.401113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] RBAC: Authorizing `identity:get_project(project_id=16218857a48d4501b2b8db4e0e5b32f7)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.402639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.404340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 224/452] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/projects/16218857a48d4501b2b8db4e0e5b32f7 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:15.409852 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.451669 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0150d2-69b0-4771-9094-8036e58e55bf None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.488890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.489547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.489829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.490137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.490972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] RBAC: Adding query filter params (group.id=e160d641702244ebbea310aed5aa4a89, role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=ae63c1ef62974692a21343d5415af4fe) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:15.491280 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.493289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.499411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 229/453] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/role_assignments?group.id=e160d641702244ebbea310aed5aa4a89&scope.project.id=ae63c1ef62974692a21343d5415af4fe&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:15.518037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27465ad2-ab68-49b7-9da0-3a68c3652a73 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.559001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44e35d-f8d7-418b-953a-b024430fdf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.559882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44e35d-f8d7-418b-953a-b024430fdf75 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.560166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44e35d-f8d7-418b-953a-b024430fdf75 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.560440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44e35d-f8d7-418b-953a-b024430fdf75 None admin] PATH_INFO: `/v3/groups/563d7d219a3e4b83a4d7fd6cbc5085b5` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.561664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44e35d-f8d7-418b-953a-b024430fdf75 None admin] RBAC: Authorizing `identity:get_group(group_id=563d7d219a3e4b83a4d7fd6cbc5085b5)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.563202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44e35d-f8d7-418b-953a-b024430fdf75 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.565110 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 225/454] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/groups/563d7d219a3e4b83a4d7fd6cbc5085b5 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:15.571372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.673150 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.689010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e3132-d5ea-4c26-8c9a-bc88f36459ab None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.732446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.733534 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.733996 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.734499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.736031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] RBAC: Adding query filter params (group.id=563d7d219a3e4b83a4d7fd6cbc5085b5, role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=16218857a48d4501b2b8db4e0e5b32f7) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:15.736599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.739233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83149cce-82fb-4166-909b-055b0e4cdb45 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.747861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 230/455] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/role_assignments?group.id=563d7d219a3e4b83a4d7fd6cbc5085b5&scope.project.id=16218857a48d4501b2b8db4e0e5b32f7&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:18.124133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b3c44-0112-4535-949c-b5d31ef359e6 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.124643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b3c44-0112-4535-949c-b5d31ef359e6 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.124643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b3c44-0112-4535-949c-b5d31ef359e6 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.126363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 226/456] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:18.134267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0bda3-7909-48c9-84c4-e8da8d24c784 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.134701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0bda3-7909-48c9-84c4-e8da8d24c784 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.135140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0bda3-7909-48c9-84c4-e8da8d24c784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.169506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c6f0bda3-7909-48c9-84c4-e8da8d24c784 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:18.171250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c6f0bda3-7909-48c9-84c4-e8da8d24c784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:18.276643 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 231/457] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:18.323442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b3c44-0112-4535-949c-b5d31ef359e6 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.417486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.467712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b3c44-0112-4535-949c-b5d31ef359e6 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.501619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5e07-d2cc-4d9d-ad19-3905043d1576 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.502293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5e07-d2cc-4d9d-ad19-3905043d1576 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.502804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5e07-d2cc-4d9d-ad19-3905043d1576 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.504615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 232/458] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:18.509921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a8e4-a0ea-45b3-ae89-202c5154ace3 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.510494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a8e4-a0ea-45b3-ae89-202c5154ace3 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.510986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a8e4-a0ea-45b3-ae89-202c5154ace3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.522105 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.523244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.523722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.524225 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.549948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f896a8e4-a0ea-45b3-ae89-202c5154ace3 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:18.551226 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f896a8e4-a0ea-45b3-ae89-202c5154ace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:18.552243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.553793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f5dd2d7b-39aa-4b98-866a-bc0504d56afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:18.573159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.574102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.574935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:18.574935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:18.574935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.574935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:18.574935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 227/459] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.575999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d2548-aad7-4114-a4d7-f9d5b29837e7 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.576287 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d2548-aad7-4114-a4d7-f9d5b29837e7 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.576559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d2548-aad7-4114-a4d7-f9d5b29837e7 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.577831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 228/460] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:18.582463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d2548-aad7-4114-a4d7-f9d5b29837e7 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.638182 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 233/461] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:18.639707 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.640140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.640691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.661408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:18.662844 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:18.674401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.730212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d2548-aad7-4114-a4d7-f9d5b29837e7 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.749313 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.753127 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 234/462] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:18.754201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.788082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.788998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.789457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.790581 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.791469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:18.791948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.793596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.807036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 229/463] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:18.808184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.878742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508154-9c88-4b4f-9974-fad88285eb47 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.915879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729e2b5-afb8-4d5b-b5ba-be91fa0e1573 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.917934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.918772 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.919054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.919323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.926701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.930747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.932620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:18.952075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.952982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.953820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:18.953820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:18.953820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.953820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:18.955524 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.956381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.956661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.956930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.958315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 235/464] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.959148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.969265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.971031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b687db8f-da34-4806-850c-85a08b0565d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:18.988466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.989329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.990226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:18.990226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:18.990226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.990226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:18.991691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 230/465] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.993142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b101e3d-8aaa-4b91-b286-023967c18cc2 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.993424 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b101e3d-8aaa-4b91-b286-023967c18cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.993694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b101e3d-8aaa-4b91-b286-023967c18cc2 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.994960 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 231/466] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:18.999825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b101e3d-8aaa-4b91-b286-023967c18cc2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.029789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.074331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250939b8-b7a9-4e72-93d1-48735e3d6453 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.105612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b101e3d-8aaa-4b91-b286-023967c18cc2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.109229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22bf12-ff61-4a53-86a6-df59c4657053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.109899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22bf12-ff61-4a53-86a6-df59c4657053 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.110211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22bf12-ff61-4a53-86a6-df59c4657053 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.110492 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22bf12-ff61-4a53-86a6-df59c4657053 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.111192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22bf12-ff61-4a53-86a6-df59c4657053 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.112611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22bf12-ff61-4a53-86a6-df59c4657053 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.139967 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.140622 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.140899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.141168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.141907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:19.142235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.143660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.155430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 232/467] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:19.156278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.225918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 236/468] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:18 2022] POST /identity/v3/users => generated 246 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:19.228327 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.228616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.228887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.249094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:19.250902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:19.254153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.272352 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4b711-e673-48ca-ab29-0a6a2b0d7202 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.310146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.317708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.321775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.321775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.321775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.321775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:19.321775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.323696 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.340514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 237/469] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:19.341465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.342618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 233/470] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:19.349375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.475046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66811445-90ca-4a7e-aa15-cb02a5711e05 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.475559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d176c-7ecf-47b7-b0b4-60ab6111dc52 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.518350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.518585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd4e57-ed36-422a-b639-5682871d4c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.519027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.519310 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.519583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.520120 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd4e57-ed36-422a-b639-5682871d4c9a None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.520299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.520808 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd4e57-ed36-422a-b639-5682871d4c9a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.521266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd4e57-ed36-422a-b639-5682871d4c9a None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.521717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.522451 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd4e57-ed36-422a-b639-5682871d4c9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.524057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd4e57-ed36-422a-b639-5682871d4c9a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.649095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 234/471] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:19 2022] POST /identity/v3/users => generated 248 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:43:19.659232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.662683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.670146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 238/472] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:19 2022] POST /identity/v3/users => generated 248 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:19.823236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60791239-5cd7-414c-96bb-8d0ecccd6409 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.877357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.878229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.878607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.878774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.896286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5a7cf-a8dd-4e6f-b7eb-9238c0114c11 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.896715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5a7cf-a8dd-4e6f-b7eb-9238c0114c11 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.896834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5a7cf-a8dd-4e6f-b7eb-9238c0114c11 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.897471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.898781 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 239/473] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:19.899135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.905046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.905342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.905608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:19.922514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:19.923431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:19.924265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:19.924265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:19.924265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:19.924265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:19.924265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 235/474] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:19.929866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.937630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:19.938906 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:19.981731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.000976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.034251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 240/475] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:20.057267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69107c48-59b4-4acf-8671-ae38c0ac46e1 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.059527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.103926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.104507 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.104787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.105059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.105795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:20.106120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.107578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.123318 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 236/476] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:20.129104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.231292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d67995-600e-4b3a-be05-a9107fcd8e89 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.297237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.297920 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.298413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.298690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.299440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.300812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.326361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80082ade-8478-43ff-ab3b-8bbc72a3d62f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.341212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.364809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.365410 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.365695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.365966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.366713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.368684 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.406976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.sql.core [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 18:43:20.406976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 18:43:20.406976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [parameters: {'id': '423faead070c401e8a0e97487a9856ae', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 18:43:20.406976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86941) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result.read() Jul 25 18:43:20.425911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:43:20.426873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application yield resource Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise self.value Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application session.commit() Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 18:43:20.427783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.session.flush() Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application compat.raise_( Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 18:43:20.428670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:43:20.430355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result.read() Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:43:20.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [parameters: {'id': '423faead070c401e8a0e97487a9856ae', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:20.432535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 18:43:20.433307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:20.435528 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 241/477] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 25 18:43:20 2022] POST /identity/v3/roles => generated 144 bytes in 377 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:20.441256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.478514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 237/478] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:20 2022] POST /identity/v3/users => generated 249 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:20.490425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.575633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06be383-775f-4aee-812f-c9866b921304 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.614980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb866c33-b907-4281-b708-cce1f576e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.616363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb866c33-b907-4281-b708-cce1f576e857 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.616796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb866c33-b907-4281-b708-cce1f576e857 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.617140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb866c33-b907-4281-b708-cce1f576e857 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.630550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb866c33-b907-4281-b708-cce1f576e857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-cb866c33-b907-4281-b708-cce1f576e857 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:20.639796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:20.640818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:20.641611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 242/479] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:20.647003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.782277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda792f6-c29b-4c6d-972f-42fae92c11df None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.818793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.819814 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.820315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.821070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.822072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:20.822571 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.824135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3c02d-6d24-476c-8a75-24b6d895afef None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.833155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 238/480] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:21.863978 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a3ddb-3be8-4cda-8f2a-93a482c07509 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.864458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a3ddb-3be8-4cda-8f2a-93a482c07509 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.864458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a3ddb-3be8-4cda-8f2a-93a482c07509 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.866080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 243/481] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:21.871508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c35ed-2149-4688-8f89-2ee51e1fb77f None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.871923 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c35ed-2149-4688-8f89-2ee51e1fb77f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.872107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c35ed-2149-4688-8f89-2ee51e1fb77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.895044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-203c35ed-2149-4688-8f89-2ee51e1fb77f None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:21.896322 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-203c35ed-2149-4688-8f89-2ee51e1fb77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:21.932232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.984391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 239/482] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:22.016406 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a3ddb-3be8-4cda-8f2a-93a482c07509 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.095594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d2ab3-120b-4c0c-9873-6ba01b0a3856 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.096219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d2ab3-120b-4c0c-9873-6ba01b0a3856 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.096693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d2ab3-120b-4c0c-9873-6ba01b0a3856 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.098600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 240/483] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:22.103545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.104050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.104523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.122797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:22.124282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:22.135159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.150453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a3ddb-3be8-4cda-8f2a-93a482c07509 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.228162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.228977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.229264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.229547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.238418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 241/484] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:22.247150 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:22.255176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.256228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.257225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:22.262030 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 244/485] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.266527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.279755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.429897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.442909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8fb9b-7a4b-41b3-ab69-fc0ba75283e6 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.460961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b46f0-db8b-4cf2-95dc-79eafe248a35 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.480493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.481030 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.481318 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.481595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.482378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:22.482773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.484265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202e2b-156d-4362-bc66-dbf41b450fbb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.494580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 242/486] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:22.495985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9b064-5685-459d-99cc-e3ec5ab4b68d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.496271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9b064-5685-459d-99cc-e3ec5ab4b68d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.496610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9b064-5685-459d-99cc-e3ec5ab4b68d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.499840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 243/487] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:22.503164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9b064-5685-459d-99cc-e3ec5ab4b68d None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.510197 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e96b7-4fc0-457b-8af5-43976a0558e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.510993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e96b7-4fc0-457b-8af5-43976a0558e6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.511269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e96b7-4fc0-457b-8af5-43976a0558e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.511536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e96b7-4fc0-457b-8af5-43976a0558e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.522581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e96b7-4fc0-457b-8af5-43976a0558e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-1c0e96b7-4fc0-457b-8af5-43976a0558e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:22.531028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.531891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:22.532668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 245/488] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:22.533861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.534064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.534339 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.547629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.553265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:22.558159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:22.623493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9b064-5685-459d-99cc-e3ec5ab4b68d None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.644204 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 246/489] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:22.645292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.688661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.689572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.689851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.690144 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.705736 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.707413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:22.724973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.725914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.726793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:22.726793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:22.726793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.726793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:22.733758 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 244/490] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.734878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.745230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.775809 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942dd3-ae6c-450c-a892-13d0c4aa5664 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.817435 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.818296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.818793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.819637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.821207 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:22.822032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.823752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.827411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.835590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 247/491] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:22.837317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.850156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523460cd-bce4-46d4-a2e0-c67b534231c9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.890706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.891674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.892588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.893095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.906366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:22.918261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.919140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.919922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:22.927458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 245/492] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.927458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.966125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85fe31-1651-4ece-b942-06557380ccde None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.999561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.007332 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.008277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.008785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.017961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.017961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.017961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.017961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.031280 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 248/493] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.032425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.041239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b9229-6560-450a-ae18-a0ed41a984e5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.077963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.078788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.079053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.079324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.091293 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.092745 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.104788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:23.112646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.117634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.118574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:23.118574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:23.118574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:23.118574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:23.118574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 246/494] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.118574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.157173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae78f-3f9f-496a-b230-515d1b33ec3d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.194757 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.196212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.196787 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.197256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.198517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.200117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.203621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.218094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 249/495] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.219009 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.223546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01d55-7dfa-415b-b69d-e8315ed14463 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.241325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.259991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.260778 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.261056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.261321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.269253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.270840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-e078de66-eee3-4572-84d2-89d15fd39c68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:23.286971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.287905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:23.288774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 247/496] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.289863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9aa30-4f68-48b1-adad-6905da03db6b None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.290106 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9aa30-4f68-48b1-adad-6905da03db6b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.290403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9aa30-4f68-48b1-adad-6905da03db6b None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.294772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 248/497] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:23.294772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9aa30-4f68-48b1-adad-6905da03db6b None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.330312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84c406-2cdc-42aa-8122-18823e47ab85 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.350066 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.363430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.366886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.366886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.366886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.366886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.366886 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.367348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.379168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 250/498] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.379963 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.396879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9aa30-4f68-48b1-adad-6905da03db6b None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.431790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.432583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.432864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.433135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.444415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.445910 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-afc3e916-de53-4e19-89e5-e3078f08fae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:23.463582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.464428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.465249 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:23.465249 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:23.465249 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:23.465249 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:23.465249 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 249/499] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.466301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.466592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.466864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.481982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c6ff9-9063-4642-8bb1-8b2ed3708744 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.484855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:23.486078 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:23.510714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.515919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.516584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.516867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.517147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.518035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.518333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.519752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1205b1-7c24-4850-8d69-46ee2d44fddd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.558588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 251/500] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.559918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dee95a-9888-42ae-94bd-b79817ddbcd1 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.560134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 250/501] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:23.560233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dee95a-9888-42ae-94bd-b79817ddbcd1 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.560475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dee95a-9888-42ae-94bd-b79817ddbcd1 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.560970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.561865 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 252/502] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:23.566399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dee95a-9888-42ae-94bd-b79817ddbcd1 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.608501 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.665974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60121e8-60dc-4466-a22b-0bfbcfc133f5 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.671975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dee95a-9888-42ae-94bd-b79817ddbcd1 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.700002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.700799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.701082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.701356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.706239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.706941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.707223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.707496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.708212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.708387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.708519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.709841 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.709950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.720830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 253/503] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.721619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-0ce35416-eb93-4a36-afed-927a52bbddb7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:23.724534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.725630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:23.726749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 251/504] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.727421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.727596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.727879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.746805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:23.747929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:23.765628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.820981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 252/505] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:23.822132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.843068 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.846992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ede88e-3a61-4081-9c02-a9f29098f17b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.882930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.883574 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.883864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.884140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.935464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cd953-8e7a-46ef-9fd4-05782082c471 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.953941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.956662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.972254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.973048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.973326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.973604 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.983794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:23.991630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.992485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:23.993272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 253/506] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.994937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.009878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.054261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.059730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 254/507] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:24.060800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.098875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac358e-dae7-4370-b113-1e0217f5e84e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.131802 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.132674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.133024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.133724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.141906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.143420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:24.158223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.159267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:24.160180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:24.160699 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d996f3d-c334-485c-bb0c-56853127ba74 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 254/508] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.194166 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.194832 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.195113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.195392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.196491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.197283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.198723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.216012 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 255/509] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.216800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.230677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.261962 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273200dd-be29-4aa5-8321-2f9a1f529236 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.273554 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.296227 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.297025 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.297328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.297587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.308829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.310817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.316979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8166b-f656-498c-bdfd-5a7622e4e9b5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:24.327959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.328834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.329664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:24.329664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:24.329664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.329664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:24.329664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 255/510] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.330227 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.355355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.356024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.356307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.356582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.357337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.357661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.359160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.367439 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 256/511] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.368553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.432160 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b53c3-b309-4bdf-a64b-6f771cf95985 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.454212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.465744 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.466831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.467125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.474421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ae528-d977-4617-ba1d-9aa0e919c5f1 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.474545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.477884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.479908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.484955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:24.497545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.498411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.499243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:24.499243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:24.499243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.499243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:24.501766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.502460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.503069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.503069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.503766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 256/512] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.503883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.504112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.504606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.505579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.524376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 257/513] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.525143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.606611 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7660a-ee91-4034-b791-5d4a5f4216bc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.626368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113cb3e-08c7-42bc-9c9a-25ccec62a7b4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.644624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.645276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.645555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.645826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.660841 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.661508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.661792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.662099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.662826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.663133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.664517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.677161 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 258/514] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.678194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.693054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.713991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.719063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.720531 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.776902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d72ff-be0d-42ee-b908-564ba82493c9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.814687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.815482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.815756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.816049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.821831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 257/515] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:24.822721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.827116 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.829563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:24.852016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.852983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.854529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:24.854529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:24.854529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.854529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:24.854529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 259/516] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.854801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.910712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.940056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa83149-2036-460f-b360-c5399d5225cb None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.947896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.961745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7d89-ce59-4112-9346-76785d2ba6a4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.977023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.977960 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.978436 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.978861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.979827 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.980219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.981865 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.997082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 258/517] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.997894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.998839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.999495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.999782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.000062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.067673 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.069059 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.097741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0298cf8-efd1-4e11-9b9c-a706f8da86c6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.131260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.131932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.132218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.132493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.133196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.135194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.143117 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.156761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.169948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 260/518] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:25.170842 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.208702 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 259/519] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:24 2022] POST /identity/v3/users => generated 247 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:25.209542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.297856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f5b90-86a8-4435-b1ce-dc8743bb0619 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.338640 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.339461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.339836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.340177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.350570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41968496-04b5-482f-95f5-80e98841c9e3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.351700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.353812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:25.374338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:25.375405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:25.376440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:25.376440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:25.376440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:25.376440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:25.376440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 261/520] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:25.376440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.395904 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.396585 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.397414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.397414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.397842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.398179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.399570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.414490 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 260/521] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.415469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.429353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.471025 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.520018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b37b3a-4a3d-4b77-a61c-a65a050e16c9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.533654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe9d3-407c-42f0-a57d-e39d7d3fce82 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.569313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.570309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.570654 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.570993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.572079 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.572980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.573333 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.573720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.579807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.581700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.584284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.585728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:25.600334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:25.601193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:25.602527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:25.604913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:25.605739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:25.607781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:25.607781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:25.607781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:25.607781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:25.608432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 262/522] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:25.608432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.610580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 261/523] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:25.612632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.747674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da045e-c75e-4835-b874-2a31c0634dc5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.748571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.750179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.752823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31686be-9b27-4590-80b6-a608bc91b119 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.790215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.791196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.791486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.791718 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.793887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.794597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.794879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.795162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.796428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.796428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.797723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.800435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.802084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.814617 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 263/524] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.814617 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:25.824092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:25.824984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:25.825947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 262/525] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:25.827278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.930076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4458a2-c5c2-4dcf-9bb6-6871312e354a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.934903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087e7b4-30fb-4b1a-938f-9340e5cdba9a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.966998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.967764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.968168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.968449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.969369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.969682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.971163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.973742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.974463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.974740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.975009 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.975734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.976031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.977398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.993877 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 263/526] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.994123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 264/527] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.994123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.995196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.018945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.023014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.131819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d362-8444-407c-aee0-7a5b13cbdc9e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.143085 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c876b8c-f67e-4394-98c5-96c6cfdae0d2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.192895 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.193623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.193912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.194215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.199519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.200217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.200500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.200808 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.201658 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.201968 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.202618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.203513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.204054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:26.219763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:26.220722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:26.226653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 264/528] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:26.227186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 265/529] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.227264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.227342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.311354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.318633 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.334098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776bbd4-d5dc-4e81-91e9-a7edf03e933a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.337118 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025e623-8eba-4115-96fc-303e11f92b2a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.368696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.369907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.370247 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.370485 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.371325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=2135d214a8274e60ac795f71a0697433) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.371637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.372746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.373083 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.373765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.374083 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.374345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.386362 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 266/530] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/role_assignments?user.id=2135d214a8274e60ac795f71a0697433&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.386362 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.392986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.395259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:26.416886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:26.417973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:26.419012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:26.422199 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 265/531] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:26.423754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.532096 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4194565-4cfd-4b7a-87d6-3aae9646f409 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.585346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.586272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.586738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.587411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.597142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25613c9c-4693-47ed-b67c-34ab0960141a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.600847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.638409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.639648 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.640068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.640450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.641630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.642052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.644088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.663387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.672765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.672908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 266/532] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.672908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.673791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.786065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd417aad-3724-4c46-82a4-94b4e28b3e7d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.804020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 267/533] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:26.804020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.834367 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.835053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.835337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.835613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.836415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.836722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.838825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.858328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 267/534] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.858328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.887823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.932249 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.953881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c669e4-f913-4200-b2e8-f00cc8a7dc4a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.973717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7aa776-da10-4ebc-b767-9d90b325f1f4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.991191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.991865 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.992430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.992714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.993554 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=aa8107f1366e487f9e2ffc84b1ffb138) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.993869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.995948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.006234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 268/535] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/role_assignments?user.id=aa8107f1366e487f9e2ffc84b1ffb138&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:27.007447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.027125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.028781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.029258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.029799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.042164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.043946 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:27.078576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:27.079525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:27.080363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:27.080363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:27.080363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:27.080363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:27.084920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 268/536] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:27.095284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.214634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3909b9-1acd-436d-a3e7-646a67e6ef31 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.222481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.241767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694977-01b3-43ea-8d66-9f26255c6c16 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.258332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.268462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.269375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.270122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.270547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.271700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=86438daf324a458aa86b9051722c4d85) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:27.272168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.274334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.282648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 269/537] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/role_assignments?user.id=86438daf324a458aa86b9051722c4d85&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:27.286068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.286068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.286068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.286068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.291887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:27.291887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.291887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb1a2-961d-4645-adab-9f506ff5af44 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.314866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 269/538] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:27.320569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.385806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4d9e4-72de-4fcd-9b7e-bf155acb1dea None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.386172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4d9e4-72de-4fcd-9b7e-bf155acb1dea None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.386397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4d9e4-72de-4fcd-9b7e-bf155acb1dea None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.388022 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 270/539] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:27.395381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.395677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.395927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.414806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:27.416594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:27.497007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7098731-c53a-496c-922e-71c181fd9926 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.538136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.539367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.539889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.540372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.554158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.557336 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.563326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 271/540] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:27.588985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:27.589869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:27.590721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:27.592761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.598275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 270/541] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:27.605001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.710582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.811315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba567eb-5a4b-41a4-bf40-80670be1b1e5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.819980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94823bf6-7e90-4049-876c-429fda24d636 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.827395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.857112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.857693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.857971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.858293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.859098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:27.859401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.860816 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.890337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 271/542] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:27.892739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.903803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.904613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.904890 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.905164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.924724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:27.932695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:27.933566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:27.934401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:27.938036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 272/543] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:27.942094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.059692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:28.075910 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098ff8-6ef8-4190-a6cd-bfe29827837a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:28.133863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918730-2a7c-4398-92e1-44448cb39cf5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:28.139054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:28.140605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.141278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.141726 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.146977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=4c21d403c2d5498e99eaf13ae72d55ef) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:28.147542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:28.149186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:28.160709 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 272/544] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/role_assignments?user.id=4c21d403c2d5498e99eaf13ae72d55ef&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:28.198176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:28.199229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.199777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.200233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.201189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:28.201682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:28.203327 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:28.218338 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 273/545] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:28.224202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.374139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe810b0-58eb-4fb8-b9eb-a84bbe942a8f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:28.425852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:28.426605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.426891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.427166 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.439013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:28.440452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-8aab5c61-b764-4420-b89a-7cbb8c7b5bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:28.464353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:28.466889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:28.468976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:28.468976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:28.468976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:28.468976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:28.468976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 273/546] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:28.471651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.501222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:28.607495 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb3be1-6c2a-4566-94a4-5d7bf56516e0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:28.659218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591947fb-a452-4034-9f49-51a9af5e21a3 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.659544 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591947fb-a452-4034-9f49-51a9af5e21a3 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.659796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591947fb-a452-4034-9f49-51a9af5e21a3 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.661201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 274/547] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:28.663655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:28.664384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.664663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.665187 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.670142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:28.670142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:28.670472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.670472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.670472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.670983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:28.691321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 274/548] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:28.698388 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:28.701553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.705941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:28.711670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:28.821553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:28.848806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce5a01-c6ce-4e6f-99ae-be016fbf9c67 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:28.859171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 275/549] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:28.894581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.922793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:28.923613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.924019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.924425 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.943308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:28.944698 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:28.974377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:28.975356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:28.976265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 275/550] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:28.991837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.013461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ec6e-4ae8-498b-a450-fb51cc50b738 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.052623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.053423 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.053703 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.053979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.061238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.083212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:29.092661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:29.093603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:29.094462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 276/551] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:29.100520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.228156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f1e80-b285-4282-b5b5-487aad2356d6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.336924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1b347-04a8-421d-9011-e7581f8390b4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.343525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.344497 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.345939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.347541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.348654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:29.349178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.350878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.411544 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.412632 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.413128 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.413603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.414597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:29.415123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.418108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.424586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 276/552] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:29.430122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.437024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.437885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 277/553] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:29.442946 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.542969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.578053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb6401-9ffc-4f4d-bef7-efeafc5581dd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.593239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dabb31-1901-40ef-89a9-6e5915e70c94 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.613065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e64d2-3d6f-4621-a4a9-9d7f8cdaaad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.613894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e64d2-3d6f-4621-a4a9-9d7f8cdaaad1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.614273 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e64d2-3d6f-4621-a4a9-9d7f8cdaaad1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.614554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e64d2-3d6f-4621-a4a9-9d7f8cdaaad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.638568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.639433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.639719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.639998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.653980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.655660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ad01e8de-60c8-41f4-bed9-ed6d17f3780c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:29.676124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:29.685628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 278/554] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1c7c1-60df-490e-8738-0f5190794705 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1c7c1-60df-490e-8738-0f5190794705 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1c7c1-60df-490e-8738-0f5190794705 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.687152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 279/555] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:29.687943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1c7c1-60df-490e-8738-0f5190794705 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.704625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e64d2-3d6f-4621-a4a9-9d7f8cdaaad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.706266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e64d2-3d6f-4621-a4a9-9d7f8cdaaad1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.709643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.800846 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1c7c1-60df-490e-8738-0f5190794705 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.813079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.825592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 277/556] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:29.827155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.827451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.827732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.844949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.845840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.846162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.846485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:29.846637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.847897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:29.848263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.849403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:29.849855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca90497-fbb7-45b7-b1c5-4582629297ad None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.875464 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 280/557] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:29.877370 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23fbd5-afa4-49b8-9fc5-0ff56679cc1a None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.877653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23fbd5-afa4-49b8-9fc5-0ff56679cc1a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.878087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23fbd5-afa4-49b8-9fc5-0ff56679cc1a None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.879655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 281/558] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:29.884631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23fbd5-afa4-49b8-9fc5-0ff56679cc1a None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.942802 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 278/559] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:29.943599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.979482 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.013469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23fbd5-afa4-49b8-9fc5-0ff56679cc1a None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.050160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.051022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.051252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.051734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c51b8-0ce2-438f-b9d8-ff196cc4e6b3 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.051896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.062791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.064572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d9ed63d9-41b3-42a8-8b5b-abdeb6199763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:30.083333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.084256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.085274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:30.085274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:30.085274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:30.085274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:30.085274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 282/560] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.086211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.086211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.086369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.096708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.097523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.097809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.098115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] PATH_INFO: `/v3/users/nova` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.104244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:30.105420 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:30.105811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.107374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:30.122989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.124099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 279/561] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.125140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.126958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.186781 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 283/562] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:30.198572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.267719 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.275110 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae32cfa-47a7-4b2d-996b-9d39d0182dce None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.312086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f2bd-07eb-48ba-abca-6e5283214a10 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.312985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.313835 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.314162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.314436 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.327513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:30.335558 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.336701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:30.337544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 280/563] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.339026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.352811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.353606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.354406 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.354702 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.355433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.355735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.357566 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.370453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 284/564] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.371331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.386643 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.463595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb74724-e459-4247-b7f1-e02b895b98a3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.479963 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf10d59-9d64-4cf8-a125-f414451bed68 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.501761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.502484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.502750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.503025 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.503864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.504171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.505763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.524962 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 281/565] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.525938 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.530496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.531618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.532401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.532891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.548087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.553124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:30.561724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.562864 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:30.563762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 285/566] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.564442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.634404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.637796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492352cb-c947-4c7d-b4ae-436e249d22d1 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.663860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0982-cc00-4629-9bb2-1b8bd31e2b08 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.671593 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.672257 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.672534 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.672806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.673550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.673850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.675255 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.683189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 282/567] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.683965 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.702514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.703344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.703630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.703908 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.710612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.712050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-7ac7e729-f7e9-4270-8f5b-ba3f09cb4e49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:30.726971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.727922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:30.728786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:30.729949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 286/568] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.731376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccddf67-814b-43b8-ad21-75957eda597d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.731662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccddf67-814b-43b8-ad21-75957eda597d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.731936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccddf67-814b-43b8-ad21-75957eda597d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.733381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 287/569] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:30.734823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccddf67-814b-43b8-ad21-75957eda597d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.757760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.783671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3dd09-7e8b-4397-b968-59b1c1ae1312 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.817606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.818464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.818731 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.819005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.829662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.831152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.840806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccddf67-814b-43b8-ad21-75957eda597d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:30.848772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.849838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.850757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:30.850757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:30.850757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:30.850757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:30.850757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 283/570] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.851037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.872048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.876984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.877649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.877934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.878256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.879065 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.879372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.881018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.888703 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 288/571] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.889486 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.952535 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3f10-c33a-4e0c-9e10-791300fd1d47 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.978588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.987393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.988231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.988479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.988755 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.990355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18d73-0fe9-42a7-a347-efde7b352383 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.999167 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.000721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-44b1ead6-cef0-4690-ab4b-9e079ef89524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:31.018658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.019630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.020515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:31.020515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:31.020515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.020515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:31.020515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 284/572] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.021232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.021341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.021569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.028958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.029629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.029911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.030229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.031053 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.031365 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.032786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.039719 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:31.040905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:31.048847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 289/573] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.049685 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.108938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.117568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 285/574] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:31.118458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.155105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a033267-0577-46e4-9840-40a81ba278ff None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.187836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.188490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.188769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.189044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.189756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.190079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.191441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.204712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 290/575] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.205516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.235804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5fee3-ecd3-49d0-beb2-14e8cc898ac3 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.245549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.269923 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.270748 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.271036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.271311 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.281842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.283442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:31.302485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.303428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.304335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:31.304335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:31.304335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.304335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:31.304335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 286/576] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.304605 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.309354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b2009-388e-40da-8cfa-246ff6d807bc None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.325301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.346193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.346855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.347137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.347409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.348131 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.348500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.349893 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.362415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 291/577] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.363670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.419458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253c90-fb6b-4655-a78f-1659c8fb3b47 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.466023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.467550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b78e2-cd14-42de-b9c1-a13326698462 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.468735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.469030 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.469307 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.470181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=67c5071ac83e4e098a84afe77ce798ac) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.470492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.471979 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.477664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 287/578] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/role_assignments?user.id=67c5071ac83e4e098a84afe77ce798ac&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.478484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.504858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.505690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.505971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.506292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.514355 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.519024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:31.527825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.528913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:31.529731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 292/579] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.530466 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.577219 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.608075 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6a8a3-a05d-40d3-a536-67f7f6461782 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.722307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900978b0-b1bc-47e5-ba04-7a78113c607d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.724680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.725487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.725770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.726070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.733364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.734895 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:31.749440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.750393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:31.751411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 288/580] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.751889 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.763195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.763867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.764158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.764433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.765164 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.765471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.766874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.779821 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 293/581] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.780656 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.848527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.864920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b313224-0f1b-46d2-9505-46dca60c406a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.891313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.900642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.902037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.903302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.903781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.904802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e864c-ef80-4d84-b28c-ebbd207d7193 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.915178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.917248 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:31.932074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.933628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:31.934619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:31.939591 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 289/582] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.939721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.950438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.951114 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.951399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.952042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.952799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.953421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.954876 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.963321 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 294/583] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.964112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.052397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fb448-437a-4b7a-90cc-48f061a52546 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.078221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a03cad-8ab2-4da5-8ae7-a60002d24843 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.091792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.092832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.093122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.093546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.094382 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.094763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.096125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.115844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 290/584] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.115844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.119613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.120038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.122177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.122492 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.122776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.130937 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.137140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:32.148348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.149286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 295/585] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.150799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.174332 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.223025 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4858-55a3-4b05-8fb7-2325460c348e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.257081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.257757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.258063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.258342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.259183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.259488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.260469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0441-f55e-4203-b214-12078ed93576 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.260967 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.279524 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 291/586] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.280316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.299556 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.301082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.301513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.301942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.312401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.314026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:32.332086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.333112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.334178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:32.334178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:32.334178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.334178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:32.335371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 296/587] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.336162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.350168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.390611 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa38a3-f990-4672-900c-9a4bf39eae85 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.408811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.427987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.428876 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.429155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.429426 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/2135d214a8274e60ac795f71a0697433/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.442938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d0d6d-e122-4fc7-a540-881394882d00 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.444883 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=2135d214a8274e60ac795f71a0697433, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.446407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.478477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.479236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.479536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.479820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.480655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.480964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.482428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.499817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 297/588] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.500666 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.518275 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 292/589] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:32 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/2135d214a8274e60ac795f71a0697433/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:32.519492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.637759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab2161-49a0-4a91-bb76-6ea8359cd324 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.668463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.668898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afbb13-aed9-46bb-b219-9417213760e6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.679233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.683184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.683184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.683184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.686332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.692967 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.694607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.714581 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:32.715515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.716490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.717356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:32.717356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:32.717356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.717356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:32.717356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 298/590] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.717862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.717989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.718429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.718894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.719838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.720312 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.722082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.735677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 293/591] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.736749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.838516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca662b-2170-44fe-abcc-4604eaacfda6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.871596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40c6f-0b37-4fb3-af12-64f88aab5656 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.876495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.877473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.877904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.878383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.892223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.893594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:32.912510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.913546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.914468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:32.914468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:32.914468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.914468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:32.914468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 299/592] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.914767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.918606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.920561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.921054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.921556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.922493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.922979 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.924529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.946094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 294/593] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.947053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.952797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.980609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.029584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa6b35-aa4c-4496-8354-3ca571dc1a36 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.063373 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a891f-f4dc-434f-a207-704054aefd47 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.068666 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.069695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.070206 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.070872 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.080199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.081646 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:33.100397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:33.101387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:33.102345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 300/594] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:33.102956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.108977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.109647 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.109922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.110223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.110947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.111245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.112654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.126884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 295/595] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.128166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.219038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.228424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd04c05-7a75-4665-ad6c-c29d6d46c94d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.247403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.258532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc8606-bb32-4344-aa98-a44988873573 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.266806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.267587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.267857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.268134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.275319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.276827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:33.291553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:33.293054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:33.293912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 301/596] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:33.294428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.304577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.305918 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.306440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.306928 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.308602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.309161 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.311251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.336829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 296/597] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.338128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.415681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c770c17-086e-429c-ad0b-ec11c14d28bd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.451989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.452800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.453058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef56b26-bcfd-4ebd-a72c-f047a9a6ac5b None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.453166 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.453346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.461461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.462954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:33.478151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:33.479139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:33.480552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:33.484829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 302/598] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:33.485599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.496966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.498061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.498541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.499021 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.500285 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.501359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.501897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.503568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.516636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 297/599] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.518038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.532545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.608324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac343952-f865-494f-8670-b715d85f5207 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.650025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.650852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.651135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.651927 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.657970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d0863-731c-406b-8ec2-13b6ba5c7026 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.664506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.665984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:33.686396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:33.687565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:33.688459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:33.688459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:33.688459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:33.688459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:33.688459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 303/600] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:33.688459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.704097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.704786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.705065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.705760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.706782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.707084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.708527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac051db1-3b7d-4709-bc35-af61dd786de7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.726390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 298/601] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.727587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49886fcc-f010-4dfb-a52a-ae252c2453a0 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.727872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49886fcc-f010-4dfb-a52a-ae252c2453a0 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.728146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49886fcc-f010-4dfb-a52a-ae252c2453a0 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.730211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 299/602] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:33.734820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49886fcc-f010-4dfb-a52a-ae252c2453a0 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.769839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.815354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d557de-b1c2-4e3f-b8b2-31ce13d7f4b7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.821179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.858366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97ca99-c61d-4549-aadf-f40d31dd0605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.859265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97ca99-c61d-4549-aadf-f40d31dd0605 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.859541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97ca99-c61d-4549-aadf-f40d31dd0605 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.859812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97ca99-c61d-4549-aadf-f40d31dd0605 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/aa8107f1366e487f9e2ffc84b1ffb138/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.861156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49886fcc-f010-4dfb-a52a-ae252c2453a0 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.877470 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97ca99-c61d-4549-aadf-f40d31dd0605 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=aa8107f1366e487f9e2ffc84b1ffb138, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.879490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97ca99-c61d-4549-aadf-f40d31dd0605 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.904860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.905966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.911271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.911271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.911271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.911271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.914944 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.932301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 300/603] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.933299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.965246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 304/604] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:33 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/aa8107f1366e487f9e2ffc84b1ffb138/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:33.979564 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.979903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.980127 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.016768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:34.018737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:34.060205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fb945-9476-4ede-806f-e40af1c819ba None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.095893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.103163 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.110039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.110039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.110039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/86438daf324a458aa86b9051722c4d85/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.110348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 305/605] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:34.111483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.123273 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=86438daf324a458aa86b9051722c4d85, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.125143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.178894 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.231422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 301/606] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:33 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/86438daf324a458aa86b9051722c4d85/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:34.232045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.255115 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b35f0-b68a-4816-a9f2-9c3cd29b0534 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.358912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.359659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.359948 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.360232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.367636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e26fd1-395d-42a9-8c75-b7ae8a2ac1b1 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.390926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.392409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.409164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.410670 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.411164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.411662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.424156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:34.426284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:34.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:34.428512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:34.431647 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 306/607] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:34.435191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:34.445207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:34.446234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:34.447128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 302/608] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:34.457337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.517072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.596039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca402-8d18-4bd7-80ec-06798ac118c5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.599141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.636059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559ce23-c39d-40d6-9f5b-b530fe3b89e2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.662738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.663422 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.663707 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.663982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.664800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:34.665091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.666574 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.673107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.673999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.674445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.674861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.675757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:34.676207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.677788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.687172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 303/609] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:34.692311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 307/610] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:34.693332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.699561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.810895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0631a-4adb-4097-81c7-a57eea876265 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.848520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d7c11-f73c-48e8-a9c9-a4b1603323dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.849559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d7c11-f73c-48e8-a9c9-a4b1603323dc None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.849939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d7c11-f73c-48e8-a9c9-a4b1603323dc None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.850391 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d7c11-f73c-48e8-a9c9-a4b1603323dc None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/4c21d403c2d5498e99eaf13ae72d55ef/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.852582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245fe4b-9245-4491-bc11-5c0a28c88130 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.855479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.866991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d7c11-f73c-48e8-a9c9-a4b1603323dc None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=4c21d403c2d5498e99eaf13ae72d55ef, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.868490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d7c11-f73c-48e8-a9c9-a4b1603323dc None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.892167 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.892886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.893163 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.893430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.905122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.906722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-331f4d53-b6c4-4528-9bb1-9e086063e0af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:34.924135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:34.925121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 304/611] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658681ca-2c6d-4634-87d2-266c3e3ee778 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658681ca-2c6d-4634-87d2-266c3e3ee778 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658681ca-2c6d-4634-87d2-266c3e3ee778 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.930452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 305/612] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:34.934839 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658681ca-2c6d-4634-87d2-266c3e3ee778 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.961778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 308/613] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:34 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/4c21d403c2d5498e99eaf13ae72d55ef/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:34.964583 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.965361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.966289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.988483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:34.990455 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:35.002704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.091178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 309/614] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:35.112704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.139861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658681ca-2c6d-4634-87d2-266c3e3ee778 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.207276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.207862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.208143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.208416 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.209146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:35.209482 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.210933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.225436 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 306/615] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:35.230760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.239473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11604233-1174-4e34-a3a3-301d9ed07687 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.247448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.280087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.281030 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.281586 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.281988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.293688 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:35.301713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:35.302877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:35.303724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:35.309423 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 310/616] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:35.310825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.313831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.346469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e079e07-6b8a-4436-bf21-465c08876fe6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.383517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.385207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.385684 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.386753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.394869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.396787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:35.412920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:35.413882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:35.414861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:35.422261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 307/617] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:35.423909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.440889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60b50-c6cf-4acb-9a56-54f672603a16 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.501157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.501765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.502072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.502355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.503193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:35.503507 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.504899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.509014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.522260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 311/618] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:35.527593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.537882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b868bf2-3337-4d6a-ae04-59f58162b6f5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.584639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.585770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.586281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.586755 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.587999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:35.588530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.590889 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.611361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 308/619] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:35.617106 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.700138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.725681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5e60c-9f5d-4ca5-b4b7-b172560f9bfe None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.756434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.762284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.763122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.763402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.763696 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.777770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.779262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:35.797294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:35.798301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:35.799870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:35.799870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:35.799870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:35.799870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:35.799870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 309/620] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:35.804265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.808736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61b23a-15ed-40a6-8a95-99cde8d24624 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.890757 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.891579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.891863 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.892131 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.906300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c59c-9529-4564-94b8-996b0726edef None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.923868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.926480 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.941904 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.942606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.942931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.943215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.943975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:35.944283 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.945655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.959186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 310/621] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:35.964547 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:35.967997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:35.968880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:35.970257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:35.970257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:35.970257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:35.970257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:35.970257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 312/622] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:35.976604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.016785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.110704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844b041-c588-4fc2-b164-2284249a0f21 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.142413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.155057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.155741 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.156024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.156299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.157205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:36.157536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.162029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.172848 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa3ec5-d38d-484f-9aaa-9c6f62ad4c57 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.173298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 313/623] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:36.178891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.248206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.249003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.249286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.249562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.261875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.263368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:36.289368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:36.290361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:36.291282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:36.294978 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 311/624] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:36.299685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.363389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0bbae-95d1-4f20-8f31-5ccab1a5fbc9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.410255 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5e549-ee70-4d27-b8b4-e7dc93770177 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.412871 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.414691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.415502 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.415784 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.416176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.424890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.426356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:36.447344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:36.450374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:36.452655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:36.453931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.453931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.453931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.453931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.458641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:36.458641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.458641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef7741-db40-4cf8-9c33-0376890f682c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.459060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 314/625] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:36.459060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.464954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 312/626] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:36.466303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46b12b-2389-4fc9-b7b7-fb4f17e81735 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.466593 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46b12b-2389-4fc9-b7b7-fb4f17e81735 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.466869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46b12b-2389-4fc9-b7b7-fb4f17e81735 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.468431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 313/627] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:36.474918 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46b12b-2389-4fc9-b7b7-fb4f17e81735 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.500761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.588795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028d13-4542-4fb5-8493-3174ef4f23d8 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.616724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46b12b-2389-4fc9-b7b7-fb4f17e81735 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.626029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.629106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.629106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.629106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.629106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:36.629106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.629966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6590a8-2a6b-4b61-95b6-34cdc693dfb8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.653463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 315/628] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:36.655244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.655532 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.655815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.657423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac541ff-b97b-47a4-a312-6ced8945100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.658878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac541ff-b97b-47a4-a312-6ced8945100d None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.659460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac541ff-b97b-47a4-a312-6ced8945100d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.660017 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac541ff-b97b-47a4-a312-6ced8945100d None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/67c5071ac83e4e098a84afe77ce798ac/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.675649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:36.676955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:36.677355 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac541ff-b97b-47a4-a312-6ced8945100d None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=67c5071ac83e4e098a84afe77ce798ac, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.679197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac541ff-b97b-47a4-a312-6ced8945100d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.750069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 316/629] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:36.750069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.774292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.782722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 314/630] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:36 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/67c5071ac83e4e098a84afe77ce798ac/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:36.791862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8c4c-6cfe-4054-bf7f-f8c9b39f8bb3 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.792938 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8c4c-6cfe-4054-bf7f-f8c9b39f8bb3 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.793282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8c4c-6cfe-4054-bf7f-f8c9b39f8bb3 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.794945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 315/631] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:36.795644 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8c4c-6cfe-4054-bf7f-f8c9b39f8bb3 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.835205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.881006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f3890-b0fa-4819-8ad6-9748a0e1ba25 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.947262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8c4c-6cfe-4054-bf7f-f8c9b39f8bb3 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.988937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08318-0f97-4d33-a2ee-22047ec42248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.990044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08318-0f97-4d33-a2ee-22047ec42248 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.990479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08318-0f97-4d33-a2ee-22047ec42248 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.990904 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08318-0f97-4d33-a2ee-22047ec42248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.004635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08318-0f97-4d33-a2ee-22047ec42248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-0ac08318-0f97-4d33-a2ee-22047ec42248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:37.013457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.014673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:37.015624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:37.018161 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 316/632] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:37.019672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.020107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.020541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.039703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:37.041289 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:37.046760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.047480 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.047765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.048042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.132318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.133508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 317/633] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:37.134196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.134791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.146515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.178636 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.264982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 317/634] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:37.266133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.288754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799f12d-6332-47f5-b395-0f2d5f3a25fa None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.326390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.327334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.327788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.328201 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.329128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.329593 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.331201 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.341015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 318/635] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.342342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.399897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f190950-1001-4fad-9b4b-844f16dcd8a4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.416629 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.437445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.453185 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.453998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.454310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.454584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.454706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9aa8f-9cfc-430d-9a3c-cb0ff971e888 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.469659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:37.480907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.482062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:37.483015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 318/636] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:37.483908 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.497716 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.498576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.498864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.499151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.510413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.511893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-74fe33f0-f6bc-43a9-9e0f-28bd38797f8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:37.530334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.531421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.532430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:37.532430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:37.532430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:37.532430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:37.532430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 319/637] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:37.533094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ad2c6-39bd-492d-934e-25f61782b97d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.533377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ad2c6-39bd-492d-934e-25f61782b97d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.533650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ad2c6-39bd-492d-934e-25f61782b97d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.538843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 320/638] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:37.539659 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ad2c6-39bd-492d-934e-25f61782b97d None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.607450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50dcd2-45dd-45cc-8e99-b1d277d2e115 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.643529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.644298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.644692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.644972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.651677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ad2c6-39bd-492d-934e-25f61782b97d None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.656412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.657942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.671179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:37.678265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.679288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:37.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:37.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:37.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:37.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 319/639] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:37.680236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.693480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.694191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.694479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.694751 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.695508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.695812 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.697175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c808ab-c37b-4807-bb2b-ccba438d9117 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.706756 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 321/640] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.708047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.708337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.708616 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.725079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.726448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:37.727628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:37.791956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181aa28-dc22-49f4-8719-bf1068bf9e7d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.810037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 322/641] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:37.810037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.836987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.837658 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.837941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.838239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.838975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.839309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.840686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.852047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 320/642] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.852854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.924329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2c2e7-72d9-4c33-9809-2dafb1f9cc7a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.936765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.961223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.962975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.963706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.964037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.964330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.965093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.965401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.966875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.968618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8821-f051-45a3-97d9-f2a919a6bb78 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.983421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 323/643] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.984360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.011156 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.011953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.012234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.012504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.024136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.025576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:38.045812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.046856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.047739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:38.047739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:38.047739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:38.047739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:38.047739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 321/644] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.048024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.098045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20002b3-d57e-4ad5-9866-fbb0f80b47ff None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.141173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.142081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.142367 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.142635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.154424 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:38.160233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bde27-9b86-4d31-9ab2-35dc096d5843 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:38.162575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.163669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:38.164566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 324/645] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.165353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.195527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.201983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.202954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.203241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.203524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.210837 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.212278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:38.227266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.228180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:38.229143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:38.233730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 322/646] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.234975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.245838 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.269610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb694d-2077-431d-8287-3068e3c9da24 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.305115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.305921 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.306237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.306510 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.314414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.315876 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:38.330481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.331475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 325/647] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.332435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.343639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161960d-b465-4d98-aaaf-db377ba76a44 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.378399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.379069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.379351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.379623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.380354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.380663 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.382036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.393779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 323/648] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.399012 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.430071 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.440234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20951b-f328-44fc-b520-68fde5d8486d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.475089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.478200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.478881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.479164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.479444 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.480247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.480554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.482057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.490806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 326/649] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.492110 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.516444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663601a6-5744-44a5-9a36-2270c2f6b3d9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.559308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.560093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.560376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.560655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.561441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.561748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.563471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.578327 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 324/650] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.583653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.604453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df05901-f356-47ff-a3ed-a31a003c6ac4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.644767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.644767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.644767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.644767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.646286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.646286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.646286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.671750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 327/651] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.671750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.684573 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.690449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842df3d-262a-40c0-a3b4-966728d45611 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.721639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.730515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.731251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.731596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.731812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.739104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.740523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:38.755302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.756257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:38.757199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:38.758154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 325/652] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.759074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.783150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09637-7c5b-4d44-8661-1939c9ae4f03 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.819276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.820081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.820369 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.820649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.832092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.833553 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:38.852562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.853556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.854832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:38.854832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:38.854832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:38.854832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:38.854832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 328/653] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.855115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.867538 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654b7a8-9ecf-4f56-bc47-81f024ac2eb3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.902595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.903253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.903537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.903813 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.904644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=2135d214a8274e60ac795f71a0697433) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.904946 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.906386 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.912656 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 326/654] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/role_assignments?user.id=2135d214a8274e60ac795f71a0697433&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.913454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.930618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.972481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.974918 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32df7d-d057-4b26-82b0-a7b1e5b59d22 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.016361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e5bfc-6665-4132-bb60-52600bd53f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.017036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e5bfc-6665-4132-bb60-52600bd53f70 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.017318 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e5bfc-6665-4132-bb60-52600bd53f70 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.017596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e5bfc-6665-4132-bb60-52600bd53f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.028853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ed53-02e7-436a-b0d9-242213566bf4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.078156 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.078846 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.079142 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.079401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.080273 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.080824 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.082052 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.095215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e5bfc-6665-4132-bb60-52600bd53f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.096699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e5bfc-6665-4132-bb60-52600bd53f70 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.101580 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 327/655] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.102450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.166932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.209463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 329/656] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:39.210665 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec94648-b42e-4e61-9a3a-aac9bef1653c None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.210950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec94648-b42e-4e61-9a3a-aac9bef1653c None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.211229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec94648-b42e-4e61-9a3a-aac9bef1653c None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.212618 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 330/657] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:39.213186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec94648-b42e-4e61-9a3a-aac9bef1653c None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.221768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044111c-4c65-4e55-8e58-d68244fd704b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.233313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.259895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.260565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.260896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.261191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.261999 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.262337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.263703 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.280679 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 328/658] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.280679 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.330995 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec94648-b42e-4e61-9a3a-aac9bef1653c None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.371550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.372264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.372465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.372741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.383351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d137af-521f-4ce8-a654-11f7b53bfe59 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.405782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.417351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.418714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.425830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.425830 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.429964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.431412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2ae5edb3-2c62-4b0b-8502-05fe650267c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:39.453170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.454147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.455039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:39.455039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:39.455039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.455039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:39.455039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 329/659] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.455904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.456007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.456293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.460278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.461905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.476333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:39.477528 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:39.496084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.566045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 330/660] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:39.566045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.569968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 331/661] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:39.570909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.640542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.681590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b2095-ebe9-4148-ab7e-f84222e020d5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.682253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11003bd-a6fa-43a6-be76-db6c7892e99d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.719062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.719849 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.720079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.720359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.720522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.720618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.720786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.721061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.721795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.722131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.723517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.728723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.730661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.735327 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 332/662] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.736211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:39.751250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.752225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:39.753162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:39.756998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 331/663] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.757638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.773539 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.846777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c905819-d27c-418a-90bd-3406c17a6490 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.862535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c9c56-b38e-4ec4-bed8-95fd73fffd8e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.873326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.887906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.888713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.893930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.893930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.899994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.901456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.903200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.904090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.904270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.904550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.915572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:39.919656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.920590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.921468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:39.921468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:39.921468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.921468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:39.924045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 333/664] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.924751 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:39.926558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.927446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:39.928306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:39.931532 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 332/665] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.932461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.020187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.054037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbe6ed-a11e-4c96-a316-197299b699a7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.055146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8e0a1-074f-4f7d-a4e6-ae4ea83a703e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.090242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.090929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.091208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.091479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.091834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.092337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.092642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.092739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.092916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.093180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.094122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.101239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.102772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.113721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 333/666] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.114597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:40.120012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:40.120957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:40.122441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:40.127894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 334/667] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:40.127894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.161840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.220463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec1012-4cd1-48f2-95b5-3ac1993ba1c6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.227300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c61a14-9fa7-4970-a8af-209d37657a84 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.251918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.258851 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.259860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.260413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.260884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.262606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.265721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.265721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.265892 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.265892 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.266171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.266449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.267217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.267520 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.268901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.276737 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 335/668] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.278095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.278814 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 334/669] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.279750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.373874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.379775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dee33-4930-4af3-9206-7df1dce1f4a4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.386497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e697224-9f09-4591-a545-577b0ce61f62 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.414074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.414726 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.415006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.415275 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.421210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.421868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.422178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.422479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.423242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.423543 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.424957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.437808 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 335/670] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.438723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.490998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.493380 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.494884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.555290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3932d-990c-445a-a5e0-43bd40f6a5a3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.592026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.592833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.593121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.593392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.605459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.606929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.610493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.617047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 336/671] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:40.618124 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:40.630740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:40.631628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:40.632517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:40.632517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:40.632517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:40.632517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:40.632517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 336/672] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:40.632797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.729414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da754-38ac-4df0-b234-ed6db4923dbd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.741159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760b0fe-2415-4fbe-b776-46d8b9552abb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.743145 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.766901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.767698 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.768037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.768311 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.775223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.777567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.777922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.778530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.778813 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.779090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.779918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=aa8107f1366e487f9e2ffc84b1ffb138) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.780228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.781636 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.795696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 337/673] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/role_assignments?user.id=aa8107f1366e487f9e2ffc84b1ffb138&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.795696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:40.795973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:40.796772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 337/674] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:40.797730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.867236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.930595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeed06-4a3f-4ae9-8010-099dd6ae19f6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.940930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b943b-78a3-47b2-8ed5-89033072e6e0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.972674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.973483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.973771 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.974655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.989437 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.990096 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.990220 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.990486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.990764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.991497 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.991606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.991802 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.993186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.006912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 338/675] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:41.007823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:41.012706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:41.013773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:41.014890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:41.014890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:41.014890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:41.014890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:41.014890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 338/676] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:41.015222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.041450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.121263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b893cfa-8f7d-487d-a9fb-652a12e21b33 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.134744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f1303-28d1-457e-894c-14d20afc449d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.138820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.157353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.158042 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.158329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.158707 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.159557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:41.159982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.161482 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.170993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.171794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.172125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.172399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.179807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 339/677] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:41.180803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.180976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.182533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-1ebc132b-6dd9-4ce3-80b9-d7c083d7416f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:41.198175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:41.199441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:41.200402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:41.200924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 339/678] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:41.202106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a0f0e-f282-4028-8327-cbd5eaea775b None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.202419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a0f0e-f282-4028-8327-cbd5eaea775b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.202705 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a0f0e-f282-4028-8327-cbd5eaea775b None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.204038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 340/679] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:41.208146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a0f0e-f282-4028-8327-cbd5eaea775b None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.288739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.302245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a72a26-0cbd-4e03-8f69-a5d9e766e96d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.313505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a0f0e-f282-4028-8327-cbd5eaea775b None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.339043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.339871 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.340463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.340580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.341299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:41.341539 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.343055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.350786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.351440 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.351728 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.352001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.352835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=86438daf324a458aa86b9051722c4d85) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:41.353145 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.354604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc539a-792b-4b3e-8ac4-1d8cd46ae606 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.357492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 340/680] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:41.358509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.364997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 341/681] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/role_assignments?user.id=86438daf324a458aa86b9051722c4d85&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:41.366516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.366807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.367086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.387721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:41.388960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:41.415100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.478118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 342/682] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:41.478118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.540076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2966011-041d-41c3-89c5-eb6393fc13a7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.576966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.590584 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.597619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.597619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.597619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.597619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:41.597619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.597619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.613207 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15e93d-5ec8-41fb-9c85-a0941ae0873b None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.619919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 341/683] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:41.620764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.658046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.658693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.659255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.659537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.670416 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.671902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:41.689183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:41.690232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:41.691782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 343/684] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:41.692320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.745024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.755193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090a202-31d5-4446-bf51-551d78e42c04 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.806509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.807315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.807612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.807963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.812752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2c9e5-7d46-4897-bb3f-c5b037e391a9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.819811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:41.828264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:41.829218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:41.830101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:41.834462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 342/685] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:41.835331 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.850770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.856873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.858001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.858590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.859014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.943986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.945820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.961467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee91e9b-7f8f-4e4f-9e51-da881fa4f9be None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.002130 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.003085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.003500 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.003882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.004847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.005256 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.006987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.022688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 343/686] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.022688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.054629 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.067937 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 344/687] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:42.069084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.104976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.151729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2307930-44c7-4bb6-9120-068560fb8b34 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.182785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf09b3-aabf-4f92-ba4b-769ad76b9b49 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.189118 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.189912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.190330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.190698 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.191554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.191954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.193538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.208275 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 344/688] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.209274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.225115 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.226169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.226601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.227022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.228129 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=4c21d403c2d5498e99eaf13ae72d55ef) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.228599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.230447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.238270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 345/689] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/role_assignments?user.id=4c21d403c2d5498e99eaf13ae72d55ef&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.239389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.352910 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.360498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b2ae8-c603-4206-88b5-9c22ae0881d6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.378320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.398895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.400487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.401102 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.401537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.413577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.415180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ee310-a75b-4cf3-b72c-20ae0915d04e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.415523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:42.437093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:42.438109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:42.439070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:42.439070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:42.439070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:42.439070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:42.441597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 346/690] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:42.449926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.469093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.469758 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.470080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.470362 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.484317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.486019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:42.505813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:42.506817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:42.508582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:42.508582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:42.508582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:42.508582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:42.508582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 345/691] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:42.513145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.627752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223492fd-b32a-432b-a733-f7a55f187868 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.644234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c16ba1-bccf-453a-837e-34b1a4271b75 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.665681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.666605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.666897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.667278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.668087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.668920 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.670606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.678290 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.685376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 347/692] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.687940 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.694252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.694252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.694252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.694252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.694252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.694610 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.694687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.707171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.709269 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 346/693] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.714735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.822649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be7811-da10-41f6-a454-077d51742a62 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.841465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d975c6e-fefc-415e-a8d8-53677a6290b5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.859245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.860046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.860328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.860603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.868818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.870276 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.879378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.880199 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.880560 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.880915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:42.885360 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:42.886323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:42.888153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 348/694] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:42.892860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.894450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.895831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:42.922614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:42.924741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:42.927033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:42.930986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 347/695] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:42.933091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.022478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.026665 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.081471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0ef48-f0e1-4748-8b40-77f33c5f2c4f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.088495 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e29cf5-8390-4922-87ae-f36c4d89b220 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.124475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.124734 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.125395 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.125509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.125676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.125932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.126071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.126418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.126822 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.127134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.127487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.127905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.128560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.129556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b226d2-a64d-4bbf-9cb0-5d87b7c6a785 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.145895 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 348/696] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:43.148349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994a469-9f9c-4b25-b6de-f6f8d6f15625 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.148739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994a469-9f9c-4b25-b6de-f6f8d6f15625 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.149115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994a469-9f9c-4b25-b6de-f6f8d6f15625 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.150352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 349/697] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:43.150653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 349/698] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:43.156137 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994a469-9f9c-4b25-b6de-f6f8d6f15625 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.156308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.280311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ed2a-70d2-4cfd-a4cd-ed11d15d53e2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.291424 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994a469-9f9c-4b25-b6de-f6f8d6f15625 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.304654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.309712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.318831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.323519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.323519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.323519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.323519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=67c5071ac83e4e098a84afe77ce798ac) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.323519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.323519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0776f9-3d3f-4a7b-99d1-a342c73a3520 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.330745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 350/699] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/role_assignments?user.id=67c5071ac83e4e098a84afe77ce798ac&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:43.332181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.332534 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.332962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.335744 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.336414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.336692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.336963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.354377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:43.357512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:43.431523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.432981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.536941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 351/700] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:43.562681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 350/701] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:43.567884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.572215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.584949 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.693740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a149-b49b-41b6-aac2-cca22ffdb1dd None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.696036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb067c8-21ac-49cf-b841-7b80a993b79b None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.717205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.735208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.736104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.736324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.736592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.737309 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.737565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.738611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.738790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.739257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.739747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.752523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.755042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:43.755159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:43.756046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:43.756966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:43.757754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 351/702] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:43.764023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:43.778760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:43.779719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:43.780616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:43.780616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:43.780616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:43.780616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:43.780616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 352/703] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:43.785692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.852044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.891077 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceedcb8-c0ac-48ca-830a-6bc48a8a1249 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.924255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e066bd2-8ca9-430d-8dcd-bbc820840e8e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.944274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.944880 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.945164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.945442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.946210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.946519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.947937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fe368-6023-4800-bc08-6b7924d2a79d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.953605 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 352/704] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:43.954985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495f57-b2cb-4d0f-8d2d-0c50f700efc5 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.955274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495f57-b2cb-4d0f-8d2d-0c50f700efc5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.955548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495f57-b2cb-4d0f-8d2d-0c50f700efc5 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.957114 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 353/705] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:43.962472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.962873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.963253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.966641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.967637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.968071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.968600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.969531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.969989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.971593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.984812 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:43.985620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 353/706] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:43.986036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:43.986962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.033599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.065131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 354/707] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:44.071112 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.130101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f61b8-4c48-4be0-9ae9-fdaf05be9973 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.166317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.174843 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.175853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.176302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.176750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.178067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.178549 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.180485 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.188870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 354/708] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:44.190118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.207570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0e11-fd01-4d83-8eab-6fb03d3b24a7 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.277342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.278726 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.279005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.279279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.288520 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.291054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:44.313931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:44.314919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:44.315799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 355/709] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:44.321668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.438242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.447959 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183021-e3ec-42e3-b43d-988f7a3e6b01 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.497888 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.499239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.499657 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.500057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.515784 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.517774 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.518769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b5564-518f-48db-bb60-3d55852b2b51 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:44.545351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:44.546329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:44.547209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:44.547209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:44.547209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:44.547209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:44.549152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 355/710] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:44.556287 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.578041 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.589071 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.589739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.598941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.598941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.598941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.598941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.598941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.621215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 356/711] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:44.630027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.757267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f69e7c-7a34-40db-bd70-542e70a28285 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.770035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0913c-146a-4ba1-a1f7-5b24c240e0be None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.800538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.801208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.801798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.801908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.802820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=2135d214a8274e60ac795f71a0697433) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.810111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.810111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.811298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 357/712] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/role_assignments?user.id=2135d214a8274e60ac795f71a0697433&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:44.830908 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d4182-2756-479b-945a-69d22819fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.831861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d4182-2756-479b-945a-69d22819fbef None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.832313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d4182-2756-479b-945a-69d22819fbef None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.832752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d4182-2756-479b-945a-69d22819fbef None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.833638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d4182-2756-479b-945a-69d22819fbef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.834110 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d4182-2756-479b-945a-69d22819fbef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.835696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d4182-2756-479b-945a-69d22819fbef None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.853829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 356/713] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:44.859363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.877484 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37071a1a-fa7d-4740-ba6d-4a9e570af869 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.878024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37071a1a-fa7d-4740-ba6d-4a9e570af869 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.878443 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37071a1a-fa7d-4740-ba6d-4a9e570af869 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.880202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 357/714] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:44.890298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275521ad-601e-41a5-b394-547f69fec884 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.890579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275521ad-601e-41a5-b394-547f69fec884 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.890852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275521ad-601e-41a5-b394-547f69fec884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.927162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-275521ad-601e-41a5-b394-547f69fec884 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:44.928377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-275521ad-601e-41a5-b394-547f69fec884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:44.934841 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.967072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.033893 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 358/715] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:45.057121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590f741-f891-4fe5-aba8-da42a22bbc5f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.093745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275521ad-601e-41a5-b394-547f69fec884 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.108974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.109700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.110073 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.110354 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.111081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.112624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.200309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275521ad-601e-41a5-b394-547f69fec884 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.236054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.236858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.237138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.237462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] PATH_INFO: `/v3/services/network` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.238221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.239621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:45.254180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:45.255236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:45.256064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 359/716] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:45.263136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.266960 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 358/717] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:44 2022] POST /identity/v3/users => generated 254 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:45.269037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.410568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fb78c-6d9d-4b20-a473-a11b7f6474ac None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.455722 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.456404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.456827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.457102 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.457821 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] RBAC: Adding query filter params (name=network) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:45.458159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.459514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e50d-1d6c-49cd-b869-77d9a3547789 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.464615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 360/718] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:45.469806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.609787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095fb2b-3443-4e90-b5e7-36bfdc10a258 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.617801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.646302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.647325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.647787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.648261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.649151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:45.649631 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.651251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a547b2-4d81-41ef-9d8a-8c324a8b19a4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.658182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 359/719] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:46.112907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c850b-5a7f-4656-a30f-a817a1075fcb None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.113806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c850b-5a7f-4656-a30f-a817a1075fcb None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.114295 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c850b-5a7f-4656-a30f-a817a1075fcb None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.116086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 361/720] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:46.121842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.122782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.123263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.146154 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:46.147681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:46.244031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 360/721] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:46.283079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c850b-5a7f-4656-a30f-a817a1075fcb None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.379701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:46.428080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c850b-5a7f-4656-a30f-a817a1075fcb None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.472568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.473736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.474243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.474753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.494891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:46.503073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:46.503983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:46.504817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:46.507597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 362/722] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:46.513608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.660057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:46.668116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626c21-a473-46a6-8a8d-02f01bd53fe9 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.717176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.718293 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.718782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.719278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.720283 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:46.720822 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:46.727562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e8558-b00a-4d59-a86c-82d72d5d3e88 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:46.737083 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 361/723] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:46.742881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.835530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a00cd-c20f-4d14-9c8d-a93e7e444939 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.836103 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a00cd-c20f-4d14-9c8d-a93e7e444939 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.836584 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a00cd-c20f-4d14-9c8d-a93e7e444939 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.838143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 362/724] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:46.853596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.853920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.854231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.872876 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:46.874566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:46.893861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601f4ec-f710-4307-b31a-56544f087639 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.933032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:46.944397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.945383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.945868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.946300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.957707 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:46.959183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:46.977857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:46.978911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:46.979803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:46.979803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:46.979803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:46.979803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:46.979803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 363/725] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:46.985403 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.989214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 363/726] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:47.030132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.097441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ad43f-7f12-4b24-9675-1eebeea1c0a7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.135603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.136398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.136570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.136803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.137572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:47.137878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.139444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:47.152496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 364/727] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:47.158626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.187184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.208419 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d7a43-c361-4e5d-985b-d94187769909 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.219219 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.286735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa7612-a8d5-43cf-b819-9a1852e97b56 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.301341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840d9e3-7f5b-48d4-abeb-5d4bcc88d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.302050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d9e3-7f5b-48d4-abeb-5d4bcc88d4fd None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.302341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d9e3-7f5b-48d4-abeb-5d4bcc88d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.302615 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d9e3-7f5b-48d4-abeb-5d4bcc88d4fd None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.303347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d9e3-7f5b-48d4-abeb-5d4bcc88d4fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.304747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d9e3-7f5b-48d4-abeb-5d4bcc88d4fd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:47.336732 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.337541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.337849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.338134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.347850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.349519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:47.366088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:47.367124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:47.368087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 365/728] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:47.373230 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.453586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 364/729] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:47 2022] POST /identity/v3/services => generated 251 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:43:47.455840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f853c3-9842-49bf-81b0-68c58b958cb1 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.456136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f853c3-9842-49bf-81b0-68c58b958cb1 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.456452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f853c3-9842-49bf-81b0-68c58b958cb1 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.458026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 365/730] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:47.463673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.464092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.464435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.494237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.498856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553dd97-25a0-4309-aa79-f52601050e42 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.511151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:47.513049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:47.555943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.557511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.558109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.558833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.560747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:47.561529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.563455 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:47.590462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 366/731] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:47.593768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.664833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 366/732] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:47.725091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.743692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25996a1-5b8f-432c-b87a-fc125a669d1e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.782662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.783682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.784148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.784603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.807915 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.833285 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.850612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bf5a4-04d6-4ab4-84a7-522e5ccbe9bb None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.861663 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.863434 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:47.887299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.888688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.888970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.889239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.902041 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:47.909896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:47.910927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:47.911750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:47.912388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 367/733] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:47.919905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.983761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 367/734] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:43:47.989479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.054188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.058396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab9dd-aaaf-4916-8d7f-a484caf89aec None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.100815 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.101821 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.102312 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.103062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.104036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.104754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.107427 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242dfa3-8dbf-4e91-bb4c-76eee0765eb6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.117691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 368/735] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:48.119447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712098-2305-4dc8-9f77-1bde5091b591 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.119920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712098-2305-4dc8-9f77-1bde5091b591 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.120406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712098-2305-4dc8-9f77-1bde5091b591 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.122357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 369/736] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:48.123248 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712098-2305-4dc8-9f77-1bde5091b591 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.134057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.142333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd0737-60c2-4c2e-99ca-5cb7d8e5c579 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.187027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.188147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.188631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.189076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.202594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.204491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b2948a9f-8955-4af7-9f4d-ad4c33561a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:48.224831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.225961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.227175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:48.227175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:48.227175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:48.227175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:48.229194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 368/737] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.230755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688b6a7-7f7d-4892-9482-babb8817f247 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.231256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688b6a7-7f7d-4892-9482-babb8817f247 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.231721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688b6a7-7f7d-4892-9482-babb8817f247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.243417 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712098-2305-4dc8-9f77-1bde5091b591 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.255535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f688b6a7-7f7d-4892-9482-babb8817f247 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:48.257123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f688b6a7-7f7d-4892-9482-babb8817f247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:48.281005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.281898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.282271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.282611 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.295188 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.296655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:48.318801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.319796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.320742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:48.320742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:48.320742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:48.320742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:48.320742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 370/738] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.321916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.342187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 369/739] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:48.343604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d1f76-e518-4b33-970e-1867cee5e424 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.343890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d1f76-e518-4b33-970e-1867cee5e424 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.344161 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d1f76-e518-4b33-970e-1867cee5e424 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.345644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 370/740] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:48.346793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d1f76-e518-4b33-970e-1867cee5e424 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.391433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.393654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.442845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dac76c-6627-45b9-afaa-778728c51f3e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.473591 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d1f76-e518-4b33-970e-1867cee5e424 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.485693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.486583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.486859 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.487130 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.487950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.488265 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.489657 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da5a28-1828-49b2-8733-e52f62a099fe None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.502454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 371/741] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:48.507751 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.508167 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.508345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.514806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.515471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.515761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.516038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.517417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.517417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.518474 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.527645 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:48.529409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:48.536228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 371/742] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:48.536228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.621113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 372/743] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:48.627382 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.646192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.652641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574e6f6-4b5f-494f-acec-076a6f5e8d20 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.682142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.701218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.702040 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.702320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.702590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.713342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:48.721212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.722398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:48.723242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 372/744] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.723975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.744117 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd70cb-9f58-4605-935f-601f0c520981 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.780986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.781954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.782379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.782745 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.789554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.791133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:48.805577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.806715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:48.807804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:48.813450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 373/745] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.814292 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.833591 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a32ae-01d1-4424-a567-a7463df46f61 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.876793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.877593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.877875 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.878185 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.886179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.887627 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:48.902253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.903998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 373/746] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.909714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.925127 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.933101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983762e-51e0-4f45-ae43-3a86dd3a8e92 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.940594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.974767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.974767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.974767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.974767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.975364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.975364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.029798 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a321b-9e77-4db2-8298-73670690a8c8 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.052517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 374/747] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:48 2022] POST /identity/v3/services => generated 240 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:49.053703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.089596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.091108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.092383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.092881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.094164 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.095112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.097278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.108953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 374/748] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.110203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.192060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90567845-d5ab-4f49-9072-cf8686927e71 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.206794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.235341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.236261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.236721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.237168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.238208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.238660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.240287 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.242906 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f1765-2e93-4445-b5ae-7cf68a3de2ec None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.254396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 375/749] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.255775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.291319 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.291660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.292370 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.292647 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.292917 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.293753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.294142 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.295578 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.315340 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 375/750] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.316134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.392929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aeb12-e9b5-45e1-b00f-fcb062ac04d7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.423069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413312-88ad-492f-8f1a-db39c83d83ea None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.432765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.433922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.434453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.434970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.446245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.448046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.454285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.460934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.461597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.461878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.462184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.463030 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=67c5071ac83e4e098a84afe77ce798ac) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.463337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.464823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.478214 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 376/751] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/role_assignments?user.id=67c5071ac83e4e098a84afe77ce798ac&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.479308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:49.602245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:49.603196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:49.604101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:49.604101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:49.604101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:49.604101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:49.611232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 376/752] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:49.613169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.683568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35efec7-f6b2-4248-824a-d5209b4b55c9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.724806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.726070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.726561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.727067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.741540 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.743422 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:49.763926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:49.764989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:49.766098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:49.766098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:49.766098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:49.766098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:49.767678 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 377/753] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:49.773237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.837826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.849160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.878528 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478c139-f226-48ea-ad26-2e0535fc15b9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.938101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.938812 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.939098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.939376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.940116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.940418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.941270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138dbf3-3a42-46c5-aba9-f3bea9084613 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.941810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1120c-b8f1-4406-bc27-118883d3fe42 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.956193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 377/754] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.957794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c5ac-51fa-4912-848c-cbc1f496965f None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.958370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c5ac-51fa-4912-848c-cbc1f496965f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.958795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c5ac-51fa-4912-848c-cbc1f496965f None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.964428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 378/755] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:49.965501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c5ac-51fa-4912-848c-cbc1f496965f None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.986563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.987351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.987760 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.988109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.988914 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.989292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.991341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c7e9e-749e-42d3-902b-0a538bb8e6bd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.009160 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 378/756] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:50.010803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.011184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.011553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.036117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:50.038046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:50.119006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c5ac-51fa-4912-848c-cbc1f496965f None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.138228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 379/757] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:50.138228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.186447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.187437 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.187907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.188644 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.199035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.201378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.203855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:50.222869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:50.224176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:50.225364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:50.227714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 379/758] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:50.228665 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.229519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.260286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc715-5050-499c-9c6a-66e85720fd56 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.299918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.301082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.301504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.301952 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.310046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.311893 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:50.327098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:50.328202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:50.329076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:50.331636 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 380/759] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:50.332690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.368477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4ad39-7d0e-411e-a1d9-d2e990d1b074 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.413375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.414133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.414420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.414706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.415728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.417420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:50.432579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:50.433578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:50.434458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 380/760] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:50.435391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.486573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.493558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4deccb-60e7-429b-a6c0-1519e5d5fda2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.520892 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.531731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.533173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.538966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.538966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.538966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:50.538966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.538966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.561884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 381/761] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:50.563747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.570796 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284a09d-f10b-447f-a2b5-2417e8632b28 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.623147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.623832 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.624120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.624402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.625250 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:50.626035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.627919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.646564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 381/762] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:50.647133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.711506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1984fcc-501b-41be-8ae8-58d26720d077 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.754736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.756601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.757417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.758269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.760152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:50.760687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.762328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.769168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 382/763] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:50.770494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.778167 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8d996-7551-4f49-aeb0-b1e1c6d85b34 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.792843 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.810893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.825217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.826529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.826813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.827359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.905593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacfd3d-67d2-4071-9faf-d6bd378af4c1 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.926989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.928997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.945422 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.946503 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.946884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.947258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/2135d214a8274e60ac795f71a0697433/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.949711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=2135d214a8274e60ac795f71a0697433, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.951972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.018267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 383/764] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:50 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/2135d214a8274e60ac795f71a0697433/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:51.022917 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.054090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 382/765] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:51.054090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.076339 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.162606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f75c-0ff0-4518-9acc-e86dd3f5f792 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.178648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.212939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31fc27-a678-4a3a-b995-da7856e472e1 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.218125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.219233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.219700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.220133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.243829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.247714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.252920 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.253609 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.254374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.254727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.255538 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.256018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.257500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.264193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 384/766] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:51.270524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f12e1538-3c81-411b-b119-e33a76a50e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:51.287232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:51.288453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:51.289797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:51.289797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:51.289797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:51.289797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:51.289797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 383/767] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:51.289797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bc1f-2027-4e18-a282-e808deaae93a None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.290288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bc1f-2027-4e18-a282-e808deaae93a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.294480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bc1f-2027-4e18-a282-e808deaae93a None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.296364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 384/768] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:51.297239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06bc1f-2027-4e18-a282-e808deaae93a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.411467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6530c80-5be9-4566-af38-8e0cc094626f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.412831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.452869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06bc1f-2027-4e18-a282-e808deaae93a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.458054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.458737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.459025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.459306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.460113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] RBAC: Adding query filter params (interface=public, service_id=8579f2cf31d849858f3e26ffd4c11034, region_id=RegionOne) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.460427 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.461809 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412699ad-f389-4365-8640-9fd0a65aa1a0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.473719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 385/769] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/endpoints?service_id=8579f2cf31d849858f3e26ffd4c11034&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:51.475523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.475817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.476113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.492750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.495162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.496125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.496608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.497123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.498148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.498655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.499286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:51.500561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.500673 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:51.515420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 385/770] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:51.516591 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.610032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 386/771] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:51.610032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.670447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff220874-0852-4bee-b138-7464fc308f55 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.706390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.707065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.707349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.707619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.708369 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.709788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.715432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 386/772] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:51.716297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.716948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.733174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7efff-ee41-4e8f-be0e-7b94b7f5dfe5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.777016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.777856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.778176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.778520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.781870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.788848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.790715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:51.808498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:51.809431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:51.810929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:51.815668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 387/773] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:51.817114 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.921807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d9d09-ecb4-4b34-901e-486079e57b53 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.965651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.967074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.967272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f767d-256d-4e42-91c2-8c4d8ae7b269 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.967379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.967597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.968243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.970299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-1a85ff02-fb9d-474a-8422-e6bfb444c4b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:51.986129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:51.987171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:51.988592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 387/774] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:51.989430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e529c5a-ebfd-4dc1-a391-7fe03c234e14 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.989430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e529c5a-ebfd-4dc1-a391-7fe03c234e14 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.989430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e529c5a-ebfd-4dc1-a391-7fe03c234e14 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.991480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 388/775] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:51.996414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.996639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.996903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.010209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.010981 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.011367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.011707 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.012568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.012940 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.015020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.017311 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:52.018847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:52.034044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 388/776] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:52.035207 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.090749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.103199 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 389/777] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:52.106277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.111027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.181071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e95a0-4b7d-4f1a-aaa3-dab625ce16fc None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.226133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.227157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.227660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.228144 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.229050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] RBAC: Adding query filter params (name=network) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.229547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.231671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.244370 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 389/778] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:52.245743 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.283731 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486fb5c-904b-4d3a-a350-9b039618af33 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.324550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.325583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.326417 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.326886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.327910 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=78f5fc7f3ad048f08a78586de85af78e) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.328428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.330681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.346431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 390/779] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/role_assignments?user.id=78f5fc7f3ad048f08a78586de85af78e&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:52.347972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.392533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010d31c-554f-450a-9b83-b9de4afb4412 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.417369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.434205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.435597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.436075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.436536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.450182 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:52.460754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:52.461833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:52.463048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:52.465786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 390/780] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:52.475682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.556626 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.629419 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75e34-cee5-458f-8b77-223eb0c939f9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.685389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b91a7-e85f-4ad1-a8f6-2680dab0e96b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.687203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.688189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.688664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.689259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.690215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] RBAC: Adding query filter params (type=network) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.690726 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.692357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.706211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 391/781] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:52.712160 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.723381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.724052 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.724329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.724598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.725353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.725650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.730051 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.737309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 391/782] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:52.743060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.789263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.906152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d73643-3f38-49d0-b02c-13d4455e1cba None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.924293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebb655-d870-4aef-8df5-8d43c0e2411d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.945862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.946849 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.947240 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.947634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.948531 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] RBAC: Adding query filter params (interface=public, service_id=b7f754f4154343a28b4064b5518298a8, region_id=RegionOne) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.948945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.950578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.959415 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 392/783] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/endpoints?service_id=b7f754f4154343a28b4064b5518298a8&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:52.959966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.964589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.997275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.998969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.998969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.999240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.013418 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.014898 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:53.036701 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:53.037839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:53.038756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:53.038756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:53.038756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:53.038756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:53.042153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 392/784] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:53.043939 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.098100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e2fea-889e-4598-962a-e149ac609c68 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.136421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.137216 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.137498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.137767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.139250 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.140016 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.150678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 393/785] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:53.171825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.179176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cbac8-087c-43d8-bd41-17db4af1defd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.219051 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.219730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.220007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.220275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.221006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:53.221307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.222707 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7328c-a531-4616-96b4-e5d16998cba5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.236815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 393/786] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:53.242634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.325475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.361145 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd49a7-7346-47d5-8ca0-52855fceac05 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.361335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd49a7-7346-47d5-8ca0-52855fceac05 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.361582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd49a7-7346-47d5-8ca0-52855fceac05 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.364171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 394/787] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:53.369711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.370001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.370303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.395795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:53.396706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:53.399542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7216fc-0a2a-418c-9492-a2ee07ae2d59 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.446429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.448375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.448857 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.449325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.460216 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.461718 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:53.477197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:53.478215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:53.479159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 394/788] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:53.484481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.490609 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 395/789] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:53.518810 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.648334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57decf91-7b8e-4448-9287-907dcc5fa907 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.679943 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.710951 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.728984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.729667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.729948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.730244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.731096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:53.731401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.732875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.743985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6594c5-032f-4dea-88f6-6f92d6b00e91 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.752189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 395/790] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:53.757727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.818839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.820766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.821051 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.821316 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.851037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:53.859854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:53.860884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:53.861727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:53.864713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 396/791] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:53.871834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.914985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23956af8-fc35-47c7-b29c-f98220d8bdcf None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.978250 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.978985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.980931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.981224 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.014874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.016331 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.041385 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f82f5-594d-4b81-9e8d-6229bc436b1b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:54.058688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:54.059637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:54.061152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:54.061152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:54.061152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:54.061152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:54.061152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 396/792] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:54.075780 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.083796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.084481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.084766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.085045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.086658 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:54.086988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.088358 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.098039 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 397/793] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:54.105244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.125285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.125913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.219396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce488e6f-a922-4dd3-aca4-778a6d01f494 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.288751 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.289425 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.289710 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.289985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.290746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:54.291059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.292437 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145809a-7223-4364-8cd5-0ce1a3b50f8a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.304686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543844-bfbe-478a-a185-1e908de2bc21 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.310182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 397/794] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:54.312247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d3b10-b346-40e5-9134-d877002dd7b5 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.312539 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d3b10-b346-40e5-9134-d877002dd7b5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.312818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d3b10-b346-40e5-9134-d877002dd7b5 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.319084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 398/795] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:54.320871 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d3b10-b346-40e5-9134-d877002dd7b5 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.349801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.351035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.351538 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.352013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.364798 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.366730 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d1e74006-a1c0-4a97-8387-3306203ff696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:54.386359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:54.387318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:54.388218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:54.388218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:54.388218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:54.388218 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:54.390576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 398/796] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:54.392082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.392594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.393071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.425612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:54.427789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.429604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:54.487809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d3b10-b346-40e5-9134-d877002dd7b5 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.539025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.564164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.564983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.565259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.565519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.575410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 399/797] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:54.576955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.582754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.584225 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:54.623801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:54.624771 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:54.625646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:54.632155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 399/798] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:54.632155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.739224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ba26-bd47-4286-84fb-ae4c6aa9cdb5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.812520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.813281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.813570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.814169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.815224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:54.815545 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.816761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7314a-dad6-4821-8cb5-14db910ba032 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.817201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.834167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 400/799] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:54.835457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.872484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.873438 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.873839 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.874145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.874802 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.876176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-008df691-53e4-4451-b862-58d036da4ab9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.888550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:54.889652 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:54.890532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 400/800] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:54.891387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d2a5-8a73-4283-a7f8-4516e2003a98 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.891387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d2a5-8a73-4283-a7f8-4516e2003a98 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.891520 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d2a5-8a73-4283-a7f8-4516e2003a98 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.892843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 401/801] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:54.897281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2d2a5-8a73-4283-a7f8-4516e2003a98 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.900005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.900831 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.955293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541e5f5-a5cb-4df6-aa0c-382646d586bd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.991387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.992042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.992324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.992597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.993804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:54.994148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.995560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.007834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 401/802] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.008033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2d2a5-8a73-4283-a7f8-4516e2003a98 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.013341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.044513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.045336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.045611 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.045881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.053897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.055370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8805543c-5a0c-4abe-9d31-ccbea9508923 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:55.069488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:55.070493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:55.071451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 402/803] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:55.072035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.072035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.072709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.090743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:55.092492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:55.125119 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67a08-96a7-4eb2-afd6-7abf8703604b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.134949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.152285 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.162886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 403/804] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:55.163186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.163685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.164283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.164729 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.165163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.166079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.166390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.167751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.172600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 402/805] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.173215 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.282719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3f70-7c33-4467-b86d-137a9c6c209e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.284316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beda981-03fb-44ba-8081-9d9460d90d05 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.318735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.319097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.319417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.319761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.319926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.320052 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.320151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.320431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/67c5071ac83e4e098a84afe77ce798ac/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.320803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.321112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.322530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=67c5071ac83e4e098a84afe77ce798ac, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.322630 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.323955 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.341203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 403/806] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.342222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.378798 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.402140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 404/807] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:55 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/67c5071ac83e4e098a84afe77ce798ac/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:55.403234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.473909 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22587ffd-c1e3-408e-874e-48f52e045ac0 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.480976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.517277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.518361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.518794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.519217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.520089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.520543 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.522264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.533301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 404/808] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.534526 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.554228 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207cdd0-0091-4b96-8094-71995f6ceccc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.596952 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.597386 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.597672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.597946 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.610104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:55.619432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:55.620383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:55.621229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 405/809] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:55.621941 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.650162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.662788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8583a5-08d0-4dc7-af57-d7d00891013a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.705716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db791e5b-4b78-469a-89ee-d1f32d1a7296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.706518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db791e5b-4b78-469a-89ee-d1f32d1a7296 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.706804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db791e5b-4b78-469a-89ee-d1f32d1a7296 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.707195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db791e5b-4b78-469a-89ee-d1f32d1a7296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.742828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7c6d-2963-40fe-a40f-843e2321e817 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.757642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.780833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db791e5b-4b78-469a-89ee-d1f32d1a7296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.782332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.782488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db791e5b-4b78-469a-89ee-d1f32d1a7296 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.783059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.783344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.783644 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.784388 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.785752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.877825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.884904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 405/810] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:55.886277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad51f-b0b3-4b13-baa1-ce68a564e186 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.886567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad51f-b0b3-4b13-baa1-ce68a564e186 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.886845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad51f-b0b3-4b13-baa1-ce68a564e186 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.890045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 406/811] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:55.890045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad51f-b0b3-4b13-baa1-ce68a564e186 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.923002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 406/812] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:55.935182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.032865 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad51f-b0b3-4b13-baa1-ce68a564e186 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.067183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4ea9-d9ea-4d79-8f5c-470b1a8079f9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.083836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.084512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.084790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.085756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.085868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:56.086115 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.087465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e06a76-b8dd-475d-9a7c-7109bb131532 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.099981 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 407/813] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:56.101341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.101629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.101902 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.116680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.117511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.117782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.119547 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.123320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:56.123320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:56.135006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.137145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:56.159801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:56.160775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:56.162243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:56.162243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:56.162243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:56.162243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:56.162243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 407/814] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:56.162520 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.188651 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.215227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 408/815] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:56.217345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.230396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.303013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c59551-fecd-4d8f-843d-a193322430b4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.338380 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088c7f1-fe62-4c5d-b6c7-9ee54f9a2ef8 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.341606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.345816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.345816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.345816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.357345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.359574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.377465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.378171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.378458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.378735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.379467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:56.379829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.381192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:56.386162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:56.387001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:56.387908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:56.387908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:56.387908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:56.387908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:56.390253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 408/816] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:56.391377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.393689 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 409/817] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:56.399038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.452633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.476540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.508062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f4c5e-3c7d-40b0-9bfd-faf0c11913df None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.523111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a73cf4-2b1a-4d7a-9b83-811338cc7896 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.549956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.550913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.551206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.551484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] PATH_INFO: `/v3/services/network` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.552159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.553538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:56.566509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:56.567516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:56.568357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:56.572779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.572927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 409/818] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:56.573707 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.574064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.574594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.575092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.576070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:56.577424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.579095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.593377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 410/819] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:56.594569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.692599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203663dc-eafd-48cc-a68c-d4cfb5ec03d9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.702412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be9427-5c65-40b8-a8b7-64d0a3942425 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.704276 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.729870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.730722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.731008 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.731288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.734274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.740289 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.741747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.742099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.743024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.743508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.743985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.744888 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] RBAC: Adding query filter params (name=network) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:56.745436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.747146 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.754856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 411/820] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:56.754856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:56.761234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:56.762233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:56.763332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:56.764458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 410/821] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:56.765377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.860217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb26abd-b42b-4bd3-80b2-a0bb2f61f474 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.878274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d9f53-f81a-478c-ad7a-a72e82c8b29a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.896615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.897542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.897895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.898266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.907050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.908833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.923535 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.924333 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.924543 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.924823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.925546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] RBAC: Adding query filter params (type=network) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:56.925852 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:56.926623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:56.928560 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:56.931383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:56.932736 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.932885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 412/822] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:56.932885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.938928 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 411/823] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:56.942859 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.959928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.008843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.035954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077c259-b62b-405e-8692-219898bf2ced None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.052585 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1655d-5ffc-4854-9250-605d73ca1508 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.071798 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.072466 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.072752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.073030 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.073825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:57.074179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.075684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.088855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 413/824] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:57.089778 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.089926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.090781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.091231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.091660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.093228 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:57.093982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.095631 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.120892 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 412/825] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:57.122189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.196068 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d39b-bb23-45fc-874a-0c53bc0ca045 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.229076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde61b6f-ef3c-4b57-8351-eac94a83b36f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.232228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.233364 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.233647 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.233927 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.234715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.236718 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.242077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.269082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.269742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.270169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.270450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.271292 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=2135d214a8274e60ac795f71a0697433) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:57.271603 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.273046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.279086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 413/826] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/role_assignments?user.id=2135d214a8274e60ac795f71a0697433&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:57.279729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.290357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.363211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 414/827] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:57.405266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aed93d-aa50-4ffd-b275-d2dd3fc40312 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.471896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.472637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.472920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.473194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.485620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.487146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-12a3e0b2-226d-456f-8bb6-0ddad961f34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:57.507288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:57.508276 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:57.509203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:57.509203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:57.509203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:57.509203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:57.509203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 414/828] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:57.515691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.614916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6690bf-45b1-45b5-935e-5124876935cf None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.615269 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6690bf-45b1-45b5-935e-5124876935cf None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.615473 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6690bf-45b1-45b5-935e-5124876935cf None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.616991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 415/829] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:57.623909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.624503 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.625145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.648902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:57.650875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:57.668235 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc539c9-0424-4a05-9237-dc4c27abaf0a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.684270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.718141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.718826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.719110 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.719384 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.720120 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:57.720913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.722556 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.739559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 415/830] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:57.745460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.748525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.762166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 416/831] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:43:57.788840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.893772 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038454fd-45bc-45f4-b92b-9fd850998c3c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.910463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d8b20-c08e-41d6-83a4-0827162c2b21 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.938936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.940080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.940632 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.940911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.947286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.948679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.948802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.949384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.949862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.950286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.962303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:57.967751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:57.968796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:57.969694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:57.970161 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 416/832] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:57.973577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:57.974548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:57.975409 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:57.976081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.976159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 417/833] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:57.986138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.010476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.050885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.094896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafaed-0288-4f06-a72c-f97f85db1e53 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.095372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a22fb-20e4-4be4-ac47-d954d33464d4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.129981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.130743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.131029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.131350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.132111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.132423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.134033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.134892 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.135897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.136452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.136926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.137913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.138593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.140884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5468a-1078-4eae-a0b0-a3cb42f1c7c4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.147912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 418/834] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:58.149065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.155951 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 417/835] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:58.162587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5239587-645d-4868-b5d9-949dd26141f9 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.162878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5239587-645d-4868-b5d9-949dd26141f9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.163155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5239587-645d-4868-b5d9-949dd26141f9 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.164593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 418/836] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:58.165140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5239587-645d-4868-b5d9-949dd26141f9 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.248591 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.269383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd75fe-7dec-49c7-aa5e-491cb5c0f606 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.292612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5239587-645d-4868-b5d9-949dd26141f9 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.307087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.307896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.308176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.308446 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.320855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.322333 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.326316 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.338523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.351303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.351303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.351303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/78f5fc7f3ad048f08a78586de85af78e/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-13b2a88f-7db5-4be7-be0f-6d6b4f6c5700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:58.351580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:58.352396 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 419/837] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.353291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.360330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=78f5fc7f3ad048f08a78586de85af78e, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.361793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.363803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:58.365023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:58.435386 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 420/838] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:58.436773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.438751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 419/839] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:58 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/78f5fc7f3ad048f08a78586de85af78e/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:58.458683 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.524944 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.591142 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fc96a-8797-4841-982f-ebc5ea3807df None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.592044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4530e33-f0de-4a02-b11d-8349b45d3f1c None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.637567 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.638588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.638908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.639728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.640097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.640465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.640766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.640950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.641236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.641502 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.642201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.642315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.643548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.655199 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 421/840] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.656411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:58.657398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:58.658303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 420/841] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:58.663576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.665123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.710136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.783118 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17c041-031e-4743-951a-f4d03b407dbd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.796171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112d14-6964-4986-afdc-e7babdc586bb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.814253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.826286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.827828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.828364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.828844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.836064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.836731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.837013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.837288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.838001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.838352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.839708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.840504 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.842988 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.849860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 421/842] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:58.851737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:58.859930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:58.860861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:58.861791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:58.864646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 422/843] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:58.872368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.960329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.963653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9559a39-726f-4668-83f6-f930b7d51418 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.983654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cbfa1-5881-42ab-8d0a-a563681ef578 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.003595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.005018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.005508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.006045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.007014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.008064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.009949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.016351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 422/844] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:59.030239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.031365 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.031810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.032234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.033326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.033926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.035732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.062168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 423/845] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:59.064807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.177621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357e197-d026-4846-8530-3ada4a245dca None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.215372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee820e1-cc22-4210-b333-07228091f5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.216026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee820e1-cc22-4210-b333-07228091f5c7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.216308 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee820e1-cc22-4210-b333-07228091f5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.216586 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee820e1-cc22-4210-b333-07228091f5c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.258518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.299319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee820e1-cc22-4210-b333-07228091f5c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.300690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee820e1-cc22-4210-b333-07228091f5c7 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.410744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 423/846] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:59.416706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.465625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.524731 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29f1a3-5d0b-4e23-a6fe-cfeba0fbba88 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.550612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec1f2-7c89-417c-a0a6-a41684623dbb None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.550839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec1f2-7c89-417c-a0a6-a41684623dbb None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.551120 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec1f2-7c89-417c-a0a6-a41684623dbb None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.552789 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 424/847] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:59.557970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.558274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.558577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.572431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.573236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.573530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.573800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.580458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:59.582491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:59.585905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.587394 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:59.607085 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:59.611597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:59.612526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:59.612526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:59.612526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:59.612526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:43:59.612526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 424/848] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:59.619624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.668244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.686587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 425/849] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:59.709284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.743458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.751241 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f92189-c412-4bae-a8f9-8bbded7778a0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.790305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.791391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.791887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.792398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.793360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.793915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.795569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.808510 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 425/850] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:59.816999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.899804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82ec30-abbc-4df4-9d4d-8e61d2a70ca8 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.939276 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.940193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.940531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.940870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.941616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.941862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d0f19-287d-446a-9ad9-1ab6f7e65dcc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.943177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:59.957671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:59.958727 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:59.959550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:43:59.964381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 426/851] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:59.966400 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.981243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.982193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.982468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.982748 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.991343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.992817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:00.009664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:00.011818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:00.013956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:00.022047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 426/852] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:00.022047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:00.029260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:00.036043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:00.075224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48f53-a403-49d7-9744-e8a2c50b7e1e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.113140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.114138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.114625 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.115086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.116017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:00.116496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:00.118145 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.128381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26833515-63ae-4b81-b289-a9023d400fd7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.128660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 427/853] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:00.130049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:00.167458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.168929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.168929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.168929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.169593 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:00.169903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:00.171982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.190124 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 427/854] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:00.192279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:00.240439 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb020bcc-c9c3-49b6-9508-cd132c3012e3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.270417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:00.273172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:00.280318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.285818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.285818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.285818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.285818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:00.285818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:00.285818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669abfd-84b5-4569-b85e-7524aeb69544 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.298171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 428/855] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:00.303401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340e118-7f8b-4384-b82c-99aa2d3b917a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.366880 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.368384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.369161 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.369885 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.371980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=67c5071ac83e4e098a84afe77ce798ac) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:00.372779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:00.375763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c9108-a1ab-4093-834a-22695169674a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.387105 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 428/856] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:00 2022] GET /identity/v3/role_assignments?user.id=67c5071ac83e4e098a84afe77ce798ac&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:00.606546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acca05-f2f1-4567-9bac-b6146ccb7017 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.608210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acca05-f2f1-4567-9bac-b6146ccb7017 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.608210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acca05-f2f1-4567-9bac-b6146ccb7017 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.608820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 429/857] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:00.614178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.614479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.614743 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.633798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:00.634963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:00.706811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 429/858] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:00.729665 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acca05-f2f1-4567-9bac-b6146ccb7017 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:00.874251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acca05-f2f1-4567-9bac-b6146ccb7017 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.913196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.913958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.914244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.914522 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.929432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:00.937922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:00.939452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:00.940921 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 430/859] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:44:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:00.945530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:01.020207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:01.056157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7900a1-caa9-487b-b125-436939da66c9 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:01.095373 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:01.096078 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.096363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.096641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.097422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:01.097735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:01.099334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098314-2b69-44f1-938e-3086168843e9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:01.110019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 430/860] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:01.115613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:01.168117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:01.224838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aea388-f07f-420c-b002-1d66d641e1f6 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.225124 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aea388-f07f-420c-b002-1d66d641e1f6 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.225407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aea388-f07f-420c-b002-1d66d641e1f6 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.227498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 431/861] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:01.230224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6075-2194-440b-9663-8dc6a2ba2a3b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:01.237285 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.237587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.237850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.257642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:01.258905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:01.269875 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:01.271246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.271726 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.272183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.287922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:01.289402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:01.308712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:01.312569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:01.315759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:01.315759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:01.315759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:01.315759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:01.315759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 431/862] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:01.316996 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:01.366035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 432/863] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:01.382243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:01.457991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:01.465705 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:01.471646 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94039fe-e5d3-46a1-af75-5dfd797ae9f8 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:01.512357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f216c1-a879-4013-8700-40253bca90e8 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:01.517648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:01.518531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.519263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.519986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.521257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:01.521820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:01.523872 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:01.547028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 432/864] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:01.549931 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:01.555607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:01.556299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.556570 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.556844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.557591 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:01.559055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:01.678109 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 433/865] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:01 2022] POST /identity/v3/services => generated 251 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:01.714969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048bb93-e27f-44c4-b65f-d51bb36a7771 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:01.773409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:01.774729 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.775124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.775567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.785929 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:01.787598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:01.810807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:01.812742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:01.814706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 433/866] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:44:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:01.822353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:01.937945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480edd62-7767-4b1f-9f56-37751e2d0e3d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:01.962703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:01.980417 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:01.981303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.981764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.982134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.983058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:01.983617 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:01.985948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.007817 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 434/867] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 25 18:44:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:02.013874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.039945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.149072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25d51-f2f9-4e56-ba67-3fb9c7b19354 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.193148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aef79-334a-4848-8656-de22977d06b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.194184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aef79-334a-4848-8656-de22977d06b4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.194663 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aef79-334a-4848-8656-de22977d06b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.195125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aef79-334a-4848-8656-de22977d06b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.270363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aef79-334a-4848-8656-de22977d06b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.272205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aef79-334a-4848-8656-de22977d06b4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.288927 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.397102 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 434/868] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:02.403059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.462371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b930-9132-43b2-9687-af387e13ae7a None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.463042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b930-9132-43b2-9687-af387e13ae7a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.463543 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b930-9132-43b2-9687-af387e13ae7a None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.465522 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 435/869] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:02.476333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.476767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.477052 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.497050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:02.498313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:02.525929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cd053-b8c3-44e7-b13a-ee2b791bbe06 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.565589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.566850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.567375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.567866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.580254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.582169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.593303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 436/870] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:02.605403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:02.606414 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:02.607315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:02.607315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:02.607315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:02.607315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:02.609860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 435/871] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:02.615789 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.619362 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.678377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.682557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.740008 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c741f1-5c3b-4e74-8ace-4c2c1501432b None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.763819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a6226-e9b0-4378-90d0-a8a95858df08 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.783294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.783903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.784140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.784405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.785141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:02.785444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.786911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce370c-a867-49b8-8b6a-d84d3dae9862 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.816279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 437/872] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:02.816279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966f34-ab50-42ab-8b57-f1b2c4fc89de None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.816279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966f34-ab50-42ab-8b57-f1b2c4fc89de None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.816279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966f34-ab50-42ab-8b57-f1b2c4fc89de None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.816279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 438/873] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:02.818858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4b9eb-83f2-4c92-8daa-f4d99b338613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.819338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966f34-ab50-42ab-8b57-f1b2c4fc89de None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.819634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4b9eb-83f2-4c92-8daa-f4d99b338613 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.819924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4b9eb-83f2-4c92-8daa-f4d99b338613 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.820190 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4b9eb-83f2-4c92-8daa-f4d99b338613 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.820915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4b9eb-83f2-4c92-8daa-f4d99b338613 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.822336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4b9eb-83f2-4c92-8daa-f4d99b338613 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.901158 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 436/874] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:02 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:02.902643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.902937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.903216 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.923952 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:02.925297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:02.954565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.957038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966f34-ab50-42ab-8b57-f1b2c4fc89de None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.022045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 437/875] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:03.024681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.025591 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.025880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.026217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.035515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.037124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.043955 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:03.061346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:03.062407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:03.063375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:03.065988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 439/876] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:03.071676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:03.082000 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:03.203782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97fa93-d6d1-44d6-9f36-68346b275243 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.213909 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28463fe4-33c5-433d-b078-7fe4e851a369 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.253110 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.253927 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.254648 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.254926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.255582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.256959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.261481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.262180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.262520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.262793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.263654 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:03.264177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.265457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:03.276634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:03.277683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:03.278564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:03.287103 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 440/878] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:03.287303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 438/878] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:03.288635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:03.289667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:03.315798 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:03.414902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:03.431728 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e36e4b-c5f6-4bcc-b638-724d1c0b7ef5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.434001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc137a-89b3-4f96-ac7f-22b2df002165 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.474630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.475695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.476142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.476562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.477667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=78f5fc7f3ad048f08a78586de85af78e) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:03.478183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.480055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.483575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.484660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.484660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.485093 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.485875 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:03.486291 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.487697 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f5d70-10fe-43b4-9c62-fb70d7ff185b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.489149 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 439/879] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:03 2022] GET /identity/v3/role_assignments?user.id=78f5fc7f3ad048f08a78586de85af78e&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:03.493371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 441/880] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:03.499639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:03.629221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a156d1-3260-4e23-83d9-d860fb47d06f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.668444 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.669129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.669413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.669689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.670444 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:03.670751 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.672181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f48678-c7e3-4092-b168-0df37665cd25 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.677930 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 440/881] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:03.965074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bef4b3-772b-4b80-8f08-b47f08a46c71 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.965313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bef4b3-772b-4b80-8f08-b47f08a46c71 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.965553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bef4b3-772b-4b80-8f08-b47f08a46c71 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.966962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 442/882] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:03.973227 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.973519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.973803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.993197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:03.994441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:04.003723 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:04.069340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 441/883] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:04.095014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bef4b3-772b-4b80-8f08-b47f08a46c71 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:04.196110 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:04.213424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bef4b3-772b-4b80-8f08-b47f08a46c71 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:04.253871 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:04.254722 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.254998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.255267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.255933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:04.257302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:04.269779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:04.271033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:04.272641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 443/884] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:04.277097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:04.385815 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba03a53-91c5-449b-94e4-e819b89977d8 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:04.427080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:04.427868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.428029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.428279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.428994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:04.429294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:04.430699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:04.435891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 442/885] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:04.441298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:04.558035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715abc-e39c-4feb-a599-602217e007d2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:04.590091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:04.591046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.591505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.591901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.598089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:04.598089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:04.598089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:04.600891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 444/886] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:04.606991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:04.622419 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:04.721484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1678b2-e529-49c7-8441-7f00a9a2587c None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:04.764222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:04.765411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.766523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.766800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.767596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] RBAC: Adding query filter params (interface=public, service_id=af99a734179f428299f8021ea2b59189, region_id=RegionOne) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:04.767909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:04.769292 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f353a-14da-4248-bdb4-8332e7523265 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:04.775364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 443/887] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/endpoints?service_id=af99a734179f428299f8021ea2b59189&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:04.783174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:04.800493 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:04.894048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91ff7-3b2b-4ea6-8ea4-6386ce8c218e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:04.935068 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae1f5-b09e-4569-8859-f1d1e6b62c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:04.935743 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae1f5-b09e-4569-8859-f1d1e6b62c0d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.936024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae1f5-b09e-4569-8859-f1d1e6b62c0d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.936365 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae1f5-b09e-4569-8859-f1d1e6b62c0d None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.937691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eae1f5-b09e-4569-8859-f1d1e6b62c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:04.939109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eae1f5-b09e-4569-8859-f1d1e6b62c0d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:04.945602 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 445/888] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:44:05.222208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0657c-1270-4b54-a87d-41903d0bea84 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.222401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0657c-1270-4b54-a87d-41903d0bea84 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.222859 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0657c-1270-4b54-a87d-41903d0bea84 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.224576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 444/889] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:05.230161 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.230626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.230800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.252382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:05.253585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:05.331318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 446/890] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:05.352911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0657c-1270-4b54-a87d-41903d0bea84 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.436240 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.470057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0657c-1270-4b54-a87d-41903d0bea84 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.507838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.508670 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.508947 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.509220 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.509886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.511310 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-996d5150-559a-426b-a80c-fc45cc4a808d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:05.524792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:05.525805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:05.526953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 445/891] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:05.532713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.574596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.668064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c0cd1-fe98-478e-889d-ba83be1752bc None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.705447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.706170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.706454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.706736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.707463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.707771 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.709158 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.712603 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fda949-7ea6-45ed-bc0b-3bcdd7e135e2 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.712891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fda949-7ea6-45ed-bc0b-3bcdd7e135e2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.713171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fda949-7ea6-45ed-bc0b-3bcdd7e135e2 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.714938 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 446/893] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:05.715063 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 447/893] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:05.721356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.721648 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.721921 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.723217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.741356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:05.742979 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:05.808074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.846412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 447/894] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:05.867131 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85dc3e-1339-4663-b180-ec4ad1993882 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.874308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.916494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.918038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.918582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.919038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.920234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.921138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.922971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.930430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 448/895] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:05.937584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.967279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:06.012418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5c2f4-b6f7-4e96-af97-fbbba79a1f95 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.051950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4da5fc-2fe6-4401-99ab-0960cc236a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.052779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4da5fc-2fe6-4401-99ab-0960cc236a5c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.053061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4da5fc-2fe6-4401-99ab-0960cc236a5c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.053335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4da5fc-2fe6-4401-99ab-0960cc236a5c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.066220 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4da5fc-2fe6-4401-99ab-0960cc236a5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:06.071940 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d00aa-8258-427a-9b42-03d14d309098 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-5a4da5fc-2fe6-4401-99ab-0960cc236a5c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:06.075045 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:06.076466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:06.077453 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 448/896] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:06.079117 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a159629-53b2-4f90-a441-0279de891a84 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.079195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a159629-53b2-4f90-a441-0279de891a84 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.079451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a159629-53b2-4f90-a441-0279de891a84 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.085574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 449/897] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:06.085758 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a159629-53b2-4f90-a441-0279de891a84 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:06.123933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.125162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.125445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.125718 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.126550 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] RBAC: Adding query filter params (interface=public, service_id=4454e79f22af4d22863acccb4777420e, region_id=RegionOne) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:06.126856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.128244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dc3af-88b0-4c37-9ddc-e8babe6bfae2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.130302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:06.136155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 449/898] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/endpoints?service_id=4454e79f22af4d22863acccb4777420e&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:06.142878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.142878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.142878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.159177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:06.160385 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:06.213926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a159629-53b2-4f90-a441-0279de891a84 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.248813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:06.249450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 450/899] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:06.250751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:06.271147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.274266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.274266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.274266 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.274779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:06.275237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.276637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.286654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 450/900] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:06.287469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:06.388319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f480f-f6fc-418d-842d-0a1c04728298 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.414788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20117c2-df7c-44f5-9a72-0ac1063a02ef None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.427671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:06.438811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.439720 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.440007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.440281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.440972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.442383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.461438 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.462371 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.462494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 451/901] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:06 2022] GET /identity/v3/services => generated 1268 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:44:06.462589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.462886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.463183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:06.463625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.465251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.563784 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 451/902] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:06 2022] POST /identity/v3/services => generated 241 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:06.617256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8e55-3e86-4321-ac0e-c784e5e1dd90 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.670415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.671553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.672346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.672623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.686085 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.687636 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:06.707896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:06.708855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:06.710257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:06.710257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:06.710257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:06.710257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:06.710257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 452/903] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:06.715473 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:06.769716 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:06.829399 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074d38d-29d7-4915-80a6-ec569bae86e7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.867753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.868435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.868720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.868995 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.869740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:06.870071 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.871464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c0adf-66bc-4ba0-86ab-13677c6c3516 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.884989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 452/904] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:06.890676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:06.943899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.011338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27aa4-c126-44c4-924b-ea81eef3eaf4 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.049237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.050468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.050960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.051436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.059462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfe941-a6fc-4660-9147-771b54506d5f None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.059814 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfe941-a6fc-4660-9147-771b54506d5f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.060004 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfe941-a6fc-4660-9147-771b54506d5f None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.061669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 453/905] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:07.062300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.064044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.067093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.067374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.067649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:07.082797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:07.083826 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:07.084773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:07.086772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:07.087735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 453/906] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:44:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:07.087966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:07.094934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.182343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 454/907] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:07.199244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.207463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.211329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9478191-2440-45ee-8d23-1a7428264fb3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.239507 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.251741 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.252512 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.252869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.253236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.254199 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:07.254601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.257444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.284134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 454/908] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:07.289897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.339375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b837867-ce24-4065-b4f6-08ef765e2ffe None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.381523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.382391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.382698 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.383123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.384271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.385867 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:07.402858 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:07.403942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:07.404809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 455/909] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:07.413747 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.439916 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725f12f-79f5-4727-9a62-bca487a0558b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.497044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.498389 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.499046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.499721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.539225 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.551082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.552983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60091e8d-df5e-416c-a5c2-b7d8d48b5747 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.599742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.600685 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.601180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.601642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.602646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:07.603146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.604845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.607611 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.609108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.616170 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 456/910] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:07.618182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.742444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 455/911] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 18:44:07.749195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.755885 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219e3ab-94cd-48d6-905c-bd7db2c04d2c None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.798326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.799011 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.799294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.799572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.800299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:07.800606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.802216 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.816693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 457/912] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:07.817715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:07.828110 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.866442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:07.885751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0e270-22a1-4109-8abf-b7805ec5354d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.947199 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392da68-e05f-4f01-9bf3-d819236ea043 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:07.952237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.953085 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.953373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.953656 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.969129 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.971363 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.990675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:07.991379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.991666 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.991940 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.992688 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:07.994087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:07.995348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:07.996229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:07.997130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:07.997130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:07.997130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:07.997130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:08.000976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 456/913] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:08.008977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.132221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 458/914] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:08.161163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb645a8-0d3c-420b-9967-67b92ff52edc None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.180223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.224957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.225815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.226139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.226487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.227984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.228351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.232078 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.247654 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 457/915] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:08.253263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.358875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.379758 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2ffa-319e-4c47-8dfb-6a6a40b54982 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.429622 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.430489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.430775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.431166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.440801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.442764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:08.460182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:08.461169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:08.462860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 459/916] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:08.467656 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.590366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6151-f8a4-471d-a15a-5af0254da7e5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.628778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.629454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.629737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.630034 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.630835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.631145 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.632594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53330fd-f0d2-42f4-8754-3eaecd2190fc None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.646067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 458/917] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.652072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.692181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168821e1-a02c-489b-a414-96c5c5303eba None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.692494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168821e1-a02c-489b-a414-96c5c5303eba None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.692742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168821e1-a02c-489b-a414-96c5c5303eba None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.694396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 459/918] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:08.701835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb943a-a639-427f-a234-4a5305125841 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.702294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb943a-a639-427f-a234-4a5305125841 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.702696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb943a-a639-427f-a234-4a5305125841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.723068 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-dacb943a-a639-427f-a234-4a5305125841 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:08.724272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-dacb943a-a639-427f-a234-4a5305125841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:08.748390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.762384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737d921-14a1-4169-bc42-e22344079147 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.808080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 460/919] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:08.812386 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.813982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.814432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.814855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.815894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] RBAC: Adding query filter params (role.id=75e31c59f5f34919abea5e57fa7d7cab, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=78f5fc7f3ad048f08a78586de85af78e) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.816348 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.818047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.826282 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6a0d7-59ec-4777-bccd-45b0becaee16 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.826470 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6a0d7-59ec-4777-bccd-45b0becaee16 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.826674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 460/920] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/role_assignments?user.id=78f5fc7f3ad048f08a78586de85af78e&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=75e31c59f5f34919abea5e57fa7d7cab => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.826791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6a0d7-59ec-4777-bccd-45b0becaee16 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.827528 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.834343 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 461/921] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:08.840467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.840753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.841037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.841361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.884505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:08.886304 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:08.963324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d1de0-413a-4cfc-a7cc-dff63e77da2a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.985119 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 462/922] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:09.007870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.021234 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.023166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.023166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.023166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.023590 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.024915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:09.042091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:09.043164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:09.043998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:09.044735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 461/923] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:09.050893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.110088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.125814 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a57a6-9554-4cfa-8b10-417da6a6472d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.146881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.169398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.170686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.171392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.172678 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.174041 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.176612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.186853 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dbd31-f51f-4374-9d2f-9f5dc0e1b9e2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:09.191297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:09.192414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:44:09.193240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:09.196467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 463/924] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:09.203586 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.234154 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.235917 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.236702 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.237407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.238939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:09.239514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.241173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.247797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 462/925] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:09.253854 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.338478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19a1d1-77a9-4519-85bf-b8c1f65f952b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.377378 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.379184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.379751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.380245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.381249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:09.381772 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.383565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.385138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8374f19-b586-4022-a7b1-89b84efb3abd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.390632 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 464/926] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:09.396242 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.438565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.443607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.444316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.444600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.444877 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.445660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:09.445966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.447438 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.451512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.456680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 463/927] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:09.466168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.527232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5a212-f62e-4119-932a-65dd8c3125a2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.574696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.575919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.576397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.576925 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.578404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:09.578931 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.581644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.589641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 465/928] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:09.596266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.617361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47499b-0313-4f5d-b512-8a1d8a9ec833 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.665457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.666781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.667069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.667340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.668142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] RBAC: Adding query filter params (interface=public, service_id=24fd6a82f3d648609ae074e3bfb2742b, region_id=RegionOne) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:09.668449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.669825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.676389 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 464/929] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/endpoints?service_id=24fd6a82f3d648609ae074e3bfb2742b&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:09.682777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.692269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.723407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab8333-d40e-4fa1-98bc-a1b8dbd93aed None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.751851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.771805 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6f663-55c8-4c8e-a27c-039a39970fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.772463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6f663-55c8-4c8e-a27c-039a39970fd1 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.772742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6f663-55c8-4c8e-a27c-039a39970fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.773011 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6f663-55c8-4c8e-a27c-039a39970fd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.773762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6f663-55c8-4c8e-a27c-039a39970fd1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.775159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6f663-55c8-4c8e-a27c-039a39970fd1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.799292 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112976d-e355-4e26-9fee-3c9051d8d06f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.843124 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f204ad-491e-4420-84e8-0410052d72e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.844916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f204ad-491e-4420-84e8-0410052d72e1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.845291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f204ad-491e-4420-84e8-0410052d72e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.846039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f204ad-491e-4420-84e8-0410052d72e1 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.847727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f204ad-491e-4420-84e8-0410052d72e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.850508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f204ad-491e-4420-84e8-0410052d72e1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.856176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 465/930] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 18:44:09.961565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 466/931] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:10.408972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593070-28a1-4c1f-846e-c85c2abea53c None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:10.409353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593070-28a1-4c1f-846e-c85c2abea53c None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:10.409549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593070-28a1-4c1f-846e-c85c2abea53c None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:10.411205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 466/932] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:10.416852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:10.417150 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:10.417429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:10.436881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:10.440104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:10.533031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 467/933] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:10.556067 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593070-28a1-4c1f-846e-c85c2abea53c None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:10.680523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593070-28a1-4c1f-846e-c85c2abea53c None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:10.724144 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:10.735725 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:10.736806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:10.740098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:10.740098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:10.740098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:10.740098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:10.743000 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING keystone.server.flask.common [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 18:44:10.803423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.sql.core [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 18:44:10.803423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 18:44:10.803423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [parameters: {'id': 'cc92bbc545c04c03a5330141d1d2e3c0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 18:44:10.803423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86942) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f6a32b87-de02-4734-be00-41f6abd50a4c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc92bbc545c04c03a5330141d1d2e3c0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc92bbc545c04c03a5330141d1d2e3c0. Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:44:10.827456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application result.read() Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 18:44:10.829811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application yield resource Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise self.value Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application session.commit() Jul 25 18:44:10.831687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self.session.flush() Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application compat.raise_( Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 18:44:10.833395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 18:44:10.845207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application result.read() Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:44:10.848217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [parameters: {'id': 'cc92bbc545c04c03a5330141d1d2e3c0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:10.851002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:10.851803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc92bbc545c04c03a5330141d1d2e3c0. Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:10.852699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 467/934] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Jul 25 18:44:10 2022] POST /identity/v3/projects => generated 262 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:10.853241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:10.917211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:10.960661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1d5d8-1088-45fe-89ef-829831129a4c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:10.996498 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:10.997621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:10.998107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.001376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.005856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.007535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:11.023171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:11.024097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:11.025013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:11.027607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 468/935] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:11.033053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a4b2e-d7ce-4017-8d53-64f7ae67f824 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.033053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a4b2e-d7ce-4017-8d53-64f7ae67f824 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.033053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a4b2e-d7ce-4017-8d53-64f7ae67f824 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.034411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.041739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 468/936] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:11.043974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.044441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.045310 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.070704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:11.072338 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:11.153601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bbc39-12e6-4fe2-927d-21d15a33d91d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.159359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 469/937] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:11.177930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.181291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.198049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.198778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.199069 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.199402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.200473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:11.200817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.202637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.227472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 469/938] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:11.316047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbcab1-096f-4c20-9b2a-e88c4391a734 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.333182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.357140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.358366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.358838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.359321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.371756 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:11.381156 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:11.382128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:11.383013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:11.385475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 470/939] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:11.392042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.502285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6aab8-4e9b-480f-8f36-3f75209c0406 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.537290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.538855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.539343 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.539837 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.540845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:11.541375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.542977 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.555857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 470/940] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:11.558885 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.666907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e0644-e91f-4d64-96dd-43a341c050da None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.702335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.703135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.703446 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.703720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.715108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.716710 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:11.734320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:11.735805 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:11.737098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:11.737098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:11.737098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:11.737098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:11.737098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 471/941] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:11.742121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.790675 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.852596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ae465-afb2-4060-a412-078eb1ffdc8a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.888879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.889545 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.889879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.890192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.890949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:11.891261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.892622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a861dd1-47ac-4065-809f-f1c177f11090 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.905210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 471/942] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:11.911237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.943522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.019623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ef47f-9332-47d7-80ee-18591a8d27af None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.060377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.064168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.064468 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.064822 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.069201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64534c-8d8d-448c-a97d-af951ad0de08 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.069484 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64534c-8d8d-448c-a97d-af951ad0de08 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.069762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64534c-8d8d-448c-a97d-af951ad0de08 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.072409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 472/943] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:12.073905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.075479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.076704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216441-9ff4-4db7-97a1-0431bab3d11b None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.076984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216441-9ff4-4db7-97a1-0431bab3d11b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.077252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216441-9ff4-4db7-97a1-0431bab3d11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:12.092794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:12.093801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.094816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 472/944] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:12.099544 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-1a216441-9ff4-4db7-97a1-0431bab3d11b None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:12.099823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.100835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1a216441-9ff4-4db7-97a1-0431bab3d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:12.186171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 473/945] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:12.194155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bebdee-5822-43f5-b789-124e0f3ee2a7 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.194579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bebdee-5822-43f5-b789-124e0f3ee2a7 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.194963 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bebdee-5822-43f5-b789-124e0f3ee2a7 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.196696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 474/946] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:12.203280 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.203711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.204092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.205034 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.217150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f994e25-bf70-4892-a43b-36e67d4e9195 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.225860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:12.227169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:12.254567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.256006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.256690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.256978 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.257257 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.258154 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.258673 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.259876 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.278036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 473/947] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:12.283561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.298195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 475/948] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:12.298195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.396357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfcfc2-6d56-44aa-bc1e-b1ff30ecc686 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.416479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfd85b-ba1b-49de-a654-cdaf2aa5ba95 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.439305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.440371 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.440828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.441248 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.442073 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.443615 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:12.457136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:12.458236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:12.459100 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.460212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 474/949] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:12.461140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.471245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.472809 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.473132 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.473382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.475071 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.490461 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.492354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:12.515661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:12.516645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:12.517589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:12.517589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:12.517589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:12.517589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:12.519508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 476/950] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:12.520562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.539985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.604063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23e709-aeb8-4919-b787-d22831935eb9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.645721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.646713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.646999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.647340 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.648006 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.649376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.660720 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9fed5-e3e3-423b-a1a0-6e9f1ade4004 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:12.666135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:12.667347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:12.668272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 475/951] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:12.669121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.711132 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.711860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.712161 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.712432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.713159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.713470 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.714876 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.721156 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 477/952] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:12.724240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.767428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.810531 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7432-07cb-4813-add0-350cf1f11be2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.834263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.850753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.851428 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.851722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.851988 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.852722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.853033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.854634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.869246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 476/953] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:12.869430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944e4e5-b689-4312-aaf6-36ebe522524d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.871261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.917934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.918713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.919001 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.920072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.920819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] RBAC: Adding query filter params (name=compute) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.921128 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.923302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.930270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 478/954] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:12.931261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.007089 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6c0dc-b1d2-4e0e-8924-2676b6bb243c None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.044548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.050907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfe1ef-d8f6-4956-8e59-bf4b03f58c9a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.058210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.058210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.058210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.058210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.058565 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:13.058565 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.058565 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.062682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 477/955] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:13.063551 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.098979 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.099806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.100087 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.100548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.109689 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.110666 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:13.127191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:13.128221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:13.129186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 479/956] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:13.129668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.133024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.200023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a3bac-4e18-47a9-9638-56de54e4f3aa None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.245326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.246092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.246325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.246606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.247335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:13.247640 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.249037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.254628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 478/957] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:13.255506 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.277344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e675d-41bd-4ed2-9d9c-160ea156625a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.321655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3bb52-2a2b-4d26-85e8-95b12dee7cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.322445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3bb52-2a2b-4d26-85e8-95b12dee7cde None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.322721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3bb52-2a2b-4d26-85e8-95b12dee7cde None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.323003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3bb52-2a2b-4d26-85e8-95b12dee7cde None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.323788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3bb52-2a2b-4d26-85e8-95b12dee7cde None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.325300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3bb52-2a2b-4d26-85e8-95b12dee7cde None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.369157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.419619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 480/958] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:13.421084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb77e5-5d88-457e-b832-b9b7eeabb3a5 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.421430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb77e5-5d88-457e-b832-b9b7eeabb3a5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.421708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb77e5-5d88-457e-b832-b9b7eeabb3a5 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.424014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 481/959] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:13.427106 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645c89-b78f-4a2d-9b3c-51af2cf9e018 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.435873 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.436516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.437167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.466650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:13.474462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:13.488836 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.489532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.489816 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.490144 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.490968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:13.491284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.492789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.507402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 479/960] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:13.513601 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.558208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.576072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 482/961] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:13.602002 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.663860 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b5afe-4748-450c-ad9a-139099cc40c3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.690506 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.702353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.703298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.703589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.704537 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/78f5fc7f3ad048f08a78586de85af78e/roles/75e31c59f5f34919abea5e57fa7d7cab` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.707451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=78f5fc7f3ad048f08a78586de85af78e, role_id=75e31c59f5f34919abea5e57fa7d7cab)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.710097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.744965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99806-06c1-47ca-a32a-db38dad6c4a9 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.788300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.788990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.789277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.789554 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.790298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.791737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.794419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WARNING keystone.server.flask.common [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 18:44:13.846538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 480/962] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:13 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/78f5fc7f3ad048f08a78586de85af78e/roles/75e31c59f5f34919abea5e57fa7d7cab => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:13.867850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.sql.core [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 18:44:13.867850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 18:44:13.867850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [parameters: {'user_id': '0f664259f7ca4930b78287d1a85375d7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 18:44:13.867850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86941) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result.read() Jul 25 18:44:13.876344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:44:13.877383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application yield resource Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise self.value Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application session.commit() Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 18:44:13.878790 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.session.flush() Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application compat.raise_( Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 18:44:13.879984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:44:13.881124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application result.read() Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:44:13.882188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [parameters: {'user_id': '0f664259f7ca4930b78287d1a85375d7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:13.883429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 18:44:13.884587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:13.885761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 483/963] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 25 18:44:13 2022] POST /identity/v3/users => generated 160 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:13.886258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:14.037420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5bdd-6224-4743-a191-d84d3c1ac0e0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:14.075491 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:14.076401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:14.076680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:14.076954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:14.086336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:14.087798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f89677ef-9828-41bf-84b3-bff3e9d002d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:14.103197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:14.106315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:14.108709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 481/964] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:14.114044 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:14.180833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:14.251157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3c95c-dd81-4e9b-90e7-9734c09c0bec None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:14.291269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:14.291877 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:14.292156 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:14.292424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:14.293245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:14.293547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:14.295201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ecff7-dc4d-4987-b671-aef80b0f4f36 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:14.335828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 484/965] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 25 18:44:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:15.507835 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aebd3-0258-4a48-98ac-1e23bb9ab735 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.508404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aebd3-0258-4a48-98ac-1e23bb9ab735 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.508404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aebd3-0258-4a48-98ac-1e23bb9ab735 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.510063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 482/966] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:15.515436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.515954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.515954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.535984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:15.537492 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:15.568443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9734184-661b-4ab3-8c64-075485f9e8a9 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.568903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9734184-661b-4ab3-8c64-075485f9e8a9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.569351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9734184-661b-4ab3-8c64-075485f9e8a9 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.571132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 483/967] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:15.578567 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.578967 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.579446 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.602793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:15.604104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:15.626262 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 485/968] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:15.642535 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.648876 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.689286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 484/969] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:15.703380 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.712785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.775296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894659-5e68-4e7b-a115-6a6341372af5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.815530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.816217 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.816501 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.816780 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.817517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.818936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.843365 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc317a2-ddcd-4c54-b012-d00e27dce92a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.886418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.887235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.887510 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.887779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.888436 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.889877 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.897542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 486/970] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:15 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-3e97bb43-275c-4fe5-b413-481df55a6336 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:15.903516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:15.904664 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:15.917994 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 485/971] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:15.918988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.967693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0385-530b-4fd9-820f-120cab6309a6 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.968298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0385-530b-4fd9-820f-120cab6309a6 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.968298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d0385-530b-4fd9-820f-120cab6309a6 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.970577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 486/972] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:15.976317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.976677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.976898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.999157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:16.000620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:16.047001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.051958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.067229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab185b-3ff0-4e9a-a3b4-be7949466c1e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.103775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 487/973] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:16.113105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.114213 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.115095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.115303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.116092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.116345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.117970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.124435 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 487/974] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:16.126773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.131769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.253846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cf20e-cd24-4000-a5c0-41a83c231f0b None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.257811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc9ce1-c3c2-4d9c-b4b8-b7915f9c6222 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.298711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.299457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.299737 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.299890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.300037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.300541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.300850 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.301092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.302659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.302659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.304653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.313157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 488/975] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:16.313334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:16.319737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:16.322301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:16.323251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:16.324088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 488/976] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:16.331099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.350576 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.356279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.441345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6c83b-5c1e-4857-b32c-0f19384d84c6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.444319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02860471-81ba-4e8c-b084-598e592eec7f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.482238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.482905 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.483205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.483476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.484281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] RBAC: Adding query filter params (interface=admin, service_id=24fd6a82f3d648609ae074e3bfb2742b, region_id=RegionOne) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.486053 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.486572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.486948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.487634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.487912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.488191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.488959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.489260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.490660 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.493693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 489/977] 10.222.0.57 () {56 vars in 1249 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/endpoints?service_id=24fd6a82f3d648609ae074e3bfb2742b&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:16.499004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.502053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 489/978] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:16.506306 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.613169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.627385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.636872 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b552-dfe2-4abf-ba6a-25937ec9f57e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.649194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc3875-346d-416a-b121-9494b96f764d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.680138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.680803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.681088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.681366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.682100 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.683565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.692243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 490/979] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/services => generated 1727 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 18:44:16.692479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.692934 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.693179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.693536 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.709642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.711210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:16.732015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:16.733206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:16.740996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:16.740996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:16.740996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:16.740996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:16.740996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 490/980] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:16.743082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.873377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0e1ab-5fd1-4155-b338-ebb6120f1c0e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.913478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.914222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.914518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.914765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.915457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.915758 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.917123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.932340 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 491/981] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:16.937724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.050319 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec31073-d09e-4e52-a399-6cd8ddd2e1c9 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.100239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.101081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.101364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.101639 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.104084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.110788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.112241 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:17.128329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:17.129359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:44:17.130383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:17.134322 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 491/982] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:17.135590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.150671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.264823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913504-1de4-4c09-ad57-a6982b459f09 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.312604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.314126 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.314554 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.314965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.316350 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:17.316659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.318953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.347429 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 492/983] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:17.349128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.474048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb89a-55e1-4723-9326-d870d9244670 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.509854 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.510545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.510835 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.511116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.581207 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.591116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.597463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.715034 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 492/984] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:17.717137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.793025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.832986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c2a49-f9ab-42b6-9d03-d356c709a11c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.871075 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.871881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.872163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.872436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.885803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.887300 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-c39cf3a7-c1be-41bc-bf59-888dab060823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:17.905163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:17.906154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:17.907123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:17.907123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:17.907123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:17.907123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:17.910081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 493/985] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:17.913000 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.020133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.028511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa546e-a7e4-4600-b181-dc3645a747d1 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.028797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa546e-a7e4-4600-b181-dc3645a747d1 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.029079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa546e-a7e4-4600-b181-dc3645a747d1 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.030644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 494/986] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:18.036740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32fc4e-9d5a-4343-a979-9ef78c19a18b None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.039599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.039599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.039599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.063269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:18.064814 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:18.089064 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.089738 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.090050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.090326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.091058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.091361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.092786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.114429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 493/987] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:18.117095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.156347 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 495/988] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:18.179866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.235366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8cfa7-0435-43a7-923e-f1159263a4e7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.239414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.276192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.277014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.277339 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.277557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.285721 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.287213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:18.302320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:18.303763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:18.305039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:18.305469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3fe68-44a0-41ec-af5f-28e5bc493922 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.305555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 494/989] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:18.309165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.326295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.350235 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.351046 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.351342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.351596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.352533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.354159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:18.369442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:18.370635 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:18.371477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 496/990] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:18.378095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.441059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea174257-63d4-4b53-8434-5960785b9d99 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.479979 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.480690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.480968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.481294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.482105 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.482411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.483842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.489207 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ef60-98ac-411d-a0f2-2e5283d99a0d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.500539 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 495/991] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:18.501630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.508052 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.531608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.532108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.532548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.533453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.534113 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] RBAC: Adding query filter params (name=compute) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.534438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.535819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.542465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 497/992] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:18.547307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.608765 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.639225 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed8e2c-4143-408f-93bb-ba301409a723 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.655956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f14fdd-2d18-40b5-b688-f3c0ee25fdff None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.688322 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.689477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.689767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.690065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.690922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] RBAC: Adding query filter params (role.id=75e31c59f5f34919abea5e57fa7d7cab, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=78f5fc7f3ad048f08a78586de85af78e) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.691280 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.692672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.694528 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.695212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.695492 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.695772 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.696571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.696885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.698290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dcf7f-48e3-4e7a-b8f6-cad3671fb2b0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.700541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 496/993] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/role_assignments?user.id=78f5fc7f3ad048f08a78586de85af78e&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=75e31c59f5f34919abea5e57fa7d7cab => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:18.708572 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 498/994] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:18.719394 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.765776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ba98-5f8b-4d13-b9b0-36b0b4fde401 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.766331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ba98-5f8b-4d13-b9b0-36b0b4fde401 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.766410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ba98-5f8b-4d13-b9b0-36b0b4fde401 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.768985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 499/995] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:18.775252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.775596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.775839 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.796513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:18.797762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:18.846401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c880c-29c6-41ef-9b3f-e4448577c773 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.851482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.883548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.884143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.884427 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.884700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.885502 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] RBAC: Adding query filter params (interface=public, service_id=82df716a1c114395ad5df1f5752b8c3f, region_id=RegionOne) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.885810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.886764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 500/996] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:18.887305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.894521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 497/997] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/endpoints?service_id=82df716a1c114395ad5df1f5752b8c3f&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:18.900272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.910423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.913734 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:19.027783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32bd22-30fc-49db-a7a0-54657384100d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:19.036386 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fed67-6970-4cb3-8921-62f383942e57 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:19.063471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:19.064768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:19.064989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:19.065527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:19.065951 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:19.067424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:19.073162 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:19.073969 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:19.074523 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:19.074631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 501/998] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:44:19.074709 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:19.075180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:19.076546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:19.090039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:19.091143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:19.092914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 498/999] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:19.098031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:19.145035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:19.215989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464de4b-a34b-400d-96d5-b913d004205f None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:19.255168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:19.255992 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:19.256467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:19.256869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:19.258123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:19.258620 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:19.261058 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:19.268606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 502/1000] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:19.274400 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:19.351345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:19.388969 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a145f-93a9-4058-9e1e-6f5f0a20d4ab None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:19.425405 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:19.425990 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:19.426305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:19.426576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:19.427395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:19.427657 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:19.428942 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0afe5-12bb-46b3-a3ea-5582c450b1eb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:19.439540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 499/1001] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:19.440753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:19.557562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:19.561029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2213f-1741-46a4-b164-8a9b3639897c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:19.598180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984fa0a-1513-4ab1-bb62-edfed6dcf73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:19.598840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984fa0a-1513-4ab1-bb62-edfed6dcf73f None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:19.599117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984fa0a-1513-4ab1-bb62-edfed6dcf73f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:19.599471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984fa0a-1513-4ab1-bb62-edfed6dcf73f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:19.600213 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984fa0a-1513-4ab1-bb62-edfed6dcf73f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:19.601571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984fa0a-1513-4ab1-bb62-edfed6dcf73f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:19.710125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 503/1002] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:20.851379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5724ac-f360-4f0d-a275-68a5a28f6a93 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:20.851902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5724ac-f360-4f0d-a275-68a5a28f6a93 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:20.851902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5724ac-f360-4f0d-a275-68a5a28f6a93 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:20.853569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 500/1003] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:20.859011 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c534c0a-80fe-4851-bc7b-dc96bb081107 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:20.859799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c534c0a-80fe-4851-bc7b-dc96bb081107 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:20.859799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c534c0a-80fe-4851-bc7b-dc96bb081107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:20.883751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0c534c0a-80fe-4851-bc7b-dc96bb081107 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:20.885002 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0c534c0a-80fe-4851-bc7b-dc96bb081107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:20.984695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 504/1004] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:21.021748 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5724ac-f360-4f0d-a275-68a5a28f6a93 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.125372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf96395-c2ce-454f-9258-0c944547f8b5 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.125577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf96395-c2ce-454f-9258-0c944547f8b5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.125845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf96395-c2ce-454f-9258-0c944547f8b5 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.127818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 505/1005] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:21.136492 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.137109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.137377 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.148516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5724ac-f360-4f0d-a275-68a5a28f6a93 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.166268 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:21.168142 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:21.185098 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.195178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.196459 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.197024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.197478 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] PATH_INFO: `/v3/services/image` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.198577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.200746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:21.221143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:21.225642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:21.226527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:21.228746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 501/1006] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:21.229588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.265671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 506/1007] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:21.288410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.309309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.347901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91c22a-0344-45f9-ac68-b50056400546 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.389455 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.390108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.390400 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.390691 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.391441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] RBAC: Adding query filter params (name=image) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.391744 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.393191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.399550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 502/1008] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:21.406146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.411098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85c86c-d3a0-427a-a611-64231b511873 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.455950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.457034 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.457370 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.457960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] PATH_INFO: `/v3/services/compute` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.458320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.459696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:21.474149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:21.475224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:21.476070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 507/1009] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:21.481391 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.481579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.528198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7629-9cde-4189-ad4f-7e0b20f61861 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.575517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.576191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.576781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.577058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.577922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] RBAC: Adding query filter params (type=image) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.578253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.579239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.579627 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.584973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 503/1010] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:21.600272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89306e2-12b7-4b9d-b414-30f40983e56b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.639210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.639957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.640187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.640469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.641283 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.641595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.643060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5f8e-7661-4957-9850-705fd704de92 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.649079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 508/1011] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:21.654886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.769061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02243e2e-cdeb-4b1d-891c-840a6a689b19 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.792589 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.804247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.804916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.805198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.805471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.806217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] RBAC: Adding query filter params (type=compute) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.812901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.812901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.812901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 504/1012] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:21.818924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.819990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78762-9472-4f76-91a0-744a00f2b088 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.820287 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78762-9472-4f76-91a0-744a00f2b088 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.820568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78762-9472-4f76-91a0-744a00f2b088 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.822426 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 509/1013] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:21.827663 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.827957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.828237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.847340 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:21.848527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:21.919591 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 510/1014] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:21.931010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53532b96-028a-41b2-bc62-846ca848c4ec None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.942829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.971134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b3b45-b98a-4d02-829d-e0c37d90cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.971921 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3b45-b98a-4d02-829d-e0c37d90cb5a None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.972464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3b45-b98a-4d02-829d-e0c37d90cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.972464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b3b45-b98a-4d02-829d-e0c37d90cb5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.973209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3b45-b98a-4d02-829d-e0c37d90cb5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.974624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b3b45-b98a-4d02-829d-e0c37d90cb5a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.023778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.049186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713eb29-7dc1-4ccb-ba02-d240e55c6b31 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.083568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.084367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.084651 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.084927 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.086250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.087695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.089337 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 511/1015] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:22.096241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.099864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 505/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:22.229189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5b11-310e-45a6-aef9-2e8f80399f6c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.287254 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494d550-6613-42fb-874e-a3278ba9639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.287936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494d550-6613-42fb-874e-a3278ba9639b None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.288223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494d550-6613-42fb-874e-a3278ba9639b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.288501 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494d550-6613-42fb-874e-a3278ba9639b None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.289208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494d550-6613-42fb-874e-a3278ba9639b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.290791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494d550-6613-42fb-874e-a3278ba9639b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.301535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.373017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 512/1017] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:22 2022] POST /identity/v3/projects => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:23.620290 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef150c-6689-46f6-a9e5-fa3e62f55a83 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:23.620807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef150c-6689-46f6-a9e5-fa3e62f55a83 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:23.620888 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef150c-6689-46f6-a9e5-fa3e62f55a83 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:23.622390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 506/1018] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:23.627406 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e2e20-7421-443a-b306-65e476ef3ebc None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:23.627825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e2e20-7421-443a-b306-65e476ef3ebc None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:23.627996 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e2e20-7421-443a-b306-65e476ef3ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:23.646617 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-036e2e20-7421-443a-b306-65e476ef3ebc None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:23.647816 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-036e2e20-7421-443a-b306-65e476ef3ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:23.719368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 513/1019] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:23 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:23.742378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef150c-6689-46f6-a9e5-fa3e62f55a83 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:23.792776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:23.852010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef150c-6689-46f6-a9e5-fa3e62f55a83 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:23.887090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b795a35-6a6a-4b50-947c-178a4deb5a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:23.887763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b795a35-6a6a-4b50-947c-178a4deb5a57 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:23.888047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b795a35-6a6a-4b50-947c-178a4deb5a57 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:23.888329 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b795a35-6a6a-4b50-947c-178a4deb5a57 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:23.889056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b795a35-6a6a-4b50-947c-178a4deb5a57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:23.890465 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b795a35-6a6a-4b50-947c-178a4deb5a57 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:24.007375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 507/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:23 2022] POST /identity/v3/services => generated 242 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:24.260080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e41c3d-de13-4232-917a-8173b9ded17b None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.260413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e41c3d-de13-4232-917a-8173b9ded17b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.260640 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e41c3d-de13-4232-917a-8173b9ded17b None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.262178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 514/1021] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:24.267299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76215480-4e81-45ed-849a-6282b0f6feca None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.267661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76215480-4e81-45ed-849a-6282b0f6feca None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.268015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76215480-4e81-45ed-849a-6282b0f6feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.286770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-76215480-4e81-45ed-849a-6282b0f6feca None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:24.287972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-76215480-4e81-45ed-849a-6282b0f6feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:24.370114 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 508/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:24.393925 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e41c3d-de13-4232-917a-8173b9ded17b None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.452637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d36d82-bc03-46ea-bcb0-f98b54e2d751 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.453640 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d36d82-bc03-46ea-bcb0-f98b54e2d751 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.453928 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d36d82-bc03-46ea-bcb0-f98b54e2d751 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.455525 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 509/1023] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:24.461221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.461711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.462203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.481333 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:24.482576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:24.508399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e41c3d-de13-4232-917a-8173b9ded17b None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.509540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:24.531404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:24.549671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.550649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.550856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.551129 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.563884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:24.566447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 510/1024] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:24.573259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:24.574588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:24.575472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 515/1025] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:24.580512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.591136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.704053 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9241e-8561-490d-9c3b-a6bc3cbf42f2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.705883 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5c5a9-8cbf-4e1d-bdf5-b50d9b05eed8 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.746654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.747722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.748213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.748718 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.749711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:24.750112 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:24.751713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:24.753898 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.755090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.755379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.755659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.756863 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:24.758932 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:24.760793 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 516/1026] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:24.762321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 511/1027] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:24.766169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.767619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.794444 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:24.824625 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:24.882203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9ce0f-ef99-429b-a9ca-d115c438fcc2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.882536 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf5aa0-af94-47cc-8b73-8534e5be5116 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.918418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.919228 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.919507 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.919775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.920432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.921194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.921475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.921860 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.922729 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:24.924686 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:24.932970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:24.934513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:24.953146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:24.961032 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:24.961975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:24.961975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:24.961975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:24.961975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:24.961975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 512/1028] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:24.961975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:25.026110 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 517/1029] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:24 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:25.047060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:25.087146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb3773-8095-4386-ae7c-717f6ef2a7ae None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:25.130804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:25.131490 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:25.131772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:25.132046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:25.132787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:25.133097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:25.134549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:25.148925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 513/1030] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:25.156365 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:25.276683 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e9899-b33b-4cd7-a7e6-44aa2e6d4da5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:25.279349 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:25.313256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:25.315303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:25.315592 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:25.316369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:25.325693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:25.327181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:25.343642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:25.344861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:25.346551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:25.350152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 518/1031] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:44:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:25.355087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:25.464984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250c19a-beb1-4a92-980e-21acd9c5b665 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:25.496116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:25.501837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:25.502587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:25.502867 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:25.503263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:25.504122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:25.504422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:25.505854 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:25.531341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 514/1032] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:44:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:25.534111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:25.659293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b62f-4ad8-4072-b5c2-79fb01e3234d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:25.697588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99586b5-c1ae-4032-9faf-0ee37a46870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:25.698266 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99586b5-c1ae-4032-9faf-0ee37a46870e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:25.698551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99586b5-c1ae-4032-9faf-0ee37a46870e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:25.698827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99586b5-c1ae-4032-9faf-0ee37a46870e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:25.736466 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:25.781505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99586b5-c1ae-4032-9faf-0ee37a46870e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:25.782991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99586b5-c1ae-4032-9faf-0ee37a46870e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:25.900889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 519/1033] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:25 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:25.909991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.024554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b15a2-cccf-445e-b401-480886faffd6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.063134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.063943 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.064230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.064514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.075940 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.077411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.087223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112cc06-d089-4ac3-8475-b79feecf942c None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.087517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112cc06-d089-4ac3-8475-b79feecf942c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.087804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112cc06-d089-4ac3-8475-b79feecf942c None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.089342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 520/1034] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:26.094476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.094766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.095042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:26.098466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:26.099351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:26.100239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:26.100239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:26.100239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:26.100239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:26.100239 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 515/1035] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:26.105431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.117238 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:26.118740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:26.155412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:26.194413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:26.204361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 521/1036] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:26.230155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.247607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122e0c8-803d-43fe-bcab-d3c3488578f4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.295781 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.297157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.297442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.298277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:26.298586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.301766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.320159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 516/1037] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:26.321087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.357369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754f940-506d-4cb4-a18d-c646622fbef5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.393882 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.394723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.395012 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.395294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] PATH_INFO: `/v3/services/image` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.395954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.397344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:26.409966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:26.411080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:26.414984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 522/1038] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:26.417244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.441153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca2893-3181-4c03-b142-145207f29eb0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.443486 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:26.486138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.487278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.487626 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.487965 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.489782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:26.495183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.496632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:26.520325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:26.521407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:26.522757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 517/1039] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:26.527862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.548281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce220c89-24b9-4543-9630-b38ed784db33 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.590666 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.591341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.591626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.591905 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.592629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] RBAC: Adding query filter params (name=image) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:26.592936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.594335 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.601177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 523/1040] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:26.607015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.701196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60c597-13a5-4c09-a86f-615afe7d76ac None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.742697 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.743298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.743571 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.743847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.744646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:26.745052 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.746563 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.747273 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d5b12-fbea-4414-bd74-da99586eaaa3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.766133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 518/1041] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:26.767770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.787663 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:26.805790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:26.807768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.808799 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.809232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.809520 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.810285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] RBAC: Adding query filter params (type=image) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:26.810595 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.811970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.819264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 524/1042] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:26.824527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:26.903692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090d32d-7296-469a-82cf-eac5165cf968 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.938098 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d22a-202f-487b-aa35-f5a3240fc499 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:26.945151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.945830 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.946112 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.946390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.947236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] RBAC: Adding query filter params (role.id=75e31c59f5f34919abea5e57fa7d7cab, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=2135d214a8274e60ac795f71a0697433) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:26.947544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.949007 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.966044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 519/1043] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/role_assignments?user.id=2135d214a8274e60ac795f71a0697433&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=75e31c59f5f34919abea5e57fa7d7cab => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:26.989622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:26.990329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:26.990616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:26.990895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:26.991713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] RBAC: Adding query filter params (interface=public, service_id=247f13067de64de6add15cec9fe0a7e9, region_id=RegionOne) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:26.992029 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:26.993536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353add8-e928-429a-acdc-fa87b50f03b1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:26.999670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 525/1044] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:26 2022] GET /identity/v3/endpoints?service_id=247f13067de64de6add15cec9fe0a7e9&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:27.006226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.116010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.135173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0315e-8598-45e0-a748-782a54d79eb5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.171645 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed249f-1928-45e5-a91a-8a3ed7ae3c29 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.171967 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed249f-1928-45e5-a91a-8a3ed7ae3c29 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.172215 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed249f-1928-45e5-a91a-8a3ed7ae3c29 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.173573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 526/1045] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:27.175028 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.175900 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.176359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.176836 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.177695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:27.179277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.179455 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:27.179553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.179858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.188962 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 520/1046] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:44:27.200452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:27.201825 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:27.259320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.284050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 527/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:27.308302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.433692 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97aa5c-aad1-47ea-b288-33053c14ba90 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.483160 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.484342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.484837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.485336 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.496819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:27.505452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:27.506649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:27.507479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:27.510169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 521/1048] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:27.515993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.656071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7b3ef-af82-4cac-9d58-f7ce0d926a96 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.700372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.704374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.704374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.704374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.704374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:27.704374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:27.704374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:27.713778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 528/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:27.719936 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.735528 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.838749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d527b-9275-42de-8777-8fa7e11abd21 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.877644 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.878554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.878840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.879116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] PATH_INFO: `/v3/users/afc03f3d11114dbbb73d2ca8e53387d5` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.894916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] RBAC: Authorizing `identity:get_user(user_id=afc03f3d11114dbbb73d2ca8e53387d5)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:27.896375 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:27.902840 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 522/1050] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/users/afc03f3d11114dbbb73d2ca8e53387d5 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:27.904441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.958834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:28.021406 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab14e9-00b2-473f-8101-894156761926 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:28.060212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:28.060212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:28.060212 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:28.060679 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:28.140525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:28.141964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:28.187841 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:28.258449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 529/1051] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:28.260466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:28.338363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:28.377397 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365fe23-27b6-4085-8db1-7068378211c0 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:28.417080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d519f0e-93d9-419b-b620-83a1b06de4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:28.417873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d519f0e-93d9-419b-b620-83a1b06de4c9 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:28.418196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d519f0e-93d9-419b-b620-83a1b06de4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:28.418473 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d519f0e-93d9-419b-b620-83a1b06de4c9 None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:28.419677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d519f0e-93d9-419b-b620-83a1b06de4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=353815900819446194fdb3eb16e81712)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:28.421116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d519f0e-93d9-419b-b620-83a1b06de4c9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:28.423134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 523/1052] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:28 2022] GET /identity/v3/projects/353815900819446194fdb3eb16e81712 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:28.429582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:28.555188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44442-0905-4a65-899f-2dc4d4e74a41 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:28.600328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:28.602724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:28.602724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:28.602724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:28.602724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=353815900819446194fdb3eb16e81712, user.id=afc03f3d11114dbbb73d2ca8e53387d5) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:28.602724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:28.604177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c005767-6f51-4398-9479-7cbe3a147e95 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:28.610454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 530/1053] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:28 2022] GET /identity/v3/role_assignments?user.id=afc03f3d11114dbbb73d2ca8e53387d5&scope.project.id=353815900819446194fdb3eb16e81712&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:29.043649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c3ba-95f7-40e4-8b14-e7af4c5a5ebc None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.044084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c3ba-95f7-40e4-8b14-e7af4c5a5ebc None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.044277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c3ba-95f7-40e4-8b14-e7af4c5a5ebc None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.045864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 524/1054] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:29.051523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa7f5d-14ce-4d98-a6f5-4e014f4dbac9 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.052024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa7f5d-14ce-4d98-a6f5-4e014f4dbac9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.052108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa7f5d-14ce-4d98-a6f5-4e014f4dbac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.066342 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:29.071535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-eeaa7f5d-14ce-4d98-a6f5-4e014f4dbac9 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:29.072916 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-eeaa7f5d-14ce-4d98-a6f5-4e014f4dbac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:29.152469 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 531/1055] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:29.174613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74c3ba-95f7-40e4-8b14-e7af4c5a5ebc None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.234061 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003a5e7-4aef-4a98-ad60-dfe025777f5b None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.234412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003a5e7-4aef-4a98-ad60-dfe025777f5b None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.234629 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003a5e7-4aef-4a98-ad60-dfe025777f5b None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.236030 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 532/1056] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:29.241584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.241862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.242234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.262867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:29.264319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:29.303775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74c3ba-95f7-40e4-8b14-e7af4c5a5ebc None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.341983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.342915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.343262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.343537 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.344571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 533/1057] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:29.347467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:29.357846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:29.366476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:29.367538 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:29.368499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:29.369294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.369379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 525/1058] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:29.373450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.447038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:29.488287 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ae0e-be50-46c7-b7e6-1698d3131331 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.498433 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e0a41-76e0-4c07-80a4-3c99b3a70c1d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.535088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.536008 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.536291 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.536679 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] PATH_INFO: `/v3/services/image` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.537341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:29.538729 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:29.547794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.548640 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.549606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.549606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.550712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:29.551237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:29.552626 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:29.559770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:29.560828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:29.561692 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 534/1059] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:29.569189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.570476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 526/1060] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:29.577562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.647477 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:29.689465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20251c-aaeb-419b-a9dd-5433ce42fc75 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.698235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104309a-7a5c-47dc-95b4-5d5ced35a796 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.727852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:29.735433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.736391 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.742969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.742969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.742969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] RBAC: Adding query filter params (name=image) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:29.742969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:29.742969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:29.744407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.745213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.745504 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.745777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.749179 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 535/1061] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:29.755325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.760049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:29.761510 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:29.779686 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:29.780689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:29.781586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:29.781586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:29.781586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:29.781586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:29.781586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 527/1062] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:29.787772 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.878827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c424-ec10-4f17-b283-e62957dd93c0 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.907573 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e8e2-a0e1-40b0-b6bf-b730d3904a80 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:29.920312 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:29.920769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.921732 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.922025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.922303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.923008 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] RBAC: Adding query filter params (type=image) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:29.923307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:29.924667 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:29.938714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 536/1063] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:29.939832 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:29.949934 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:29.950649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:29.950939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:29.951215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:29.952086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:29.952420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:29.954079 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:29.968247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 528/1064] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:29.973983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.023276 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.077943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647235a2-dc15-48e2-b19d-069d0b21a67b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.092634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980715bc-1ec3-4897-882e-804ecb5b0729 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.116855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.117573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.117862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.118171 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.118915 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.120282 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.133843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.134609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.135011 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.135291 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.142820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.144305 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:30.159828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:30.160896 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:30.161856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 529/1065] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:44:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:30.167062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.199347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.208904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 537/1066] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:30.291889 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96066f1-07a3-4324-8aa5-cda5ac0884b2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.338714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.339393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.339680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.339937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.340829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:30.341137 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.343612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.371582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 530/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:30.378523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.508323 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32eb90-18e7-4830-92e1-fd91d4a516e2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.523369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.548875 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.549753 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.550063 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.550353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.566053 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.567710 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-56f6616d-fd87-4618-8022-67f09e2e88c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:30.586773 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:30.587745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:30.588792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:30.588792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:30.588792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:30.588792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:30.593176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 538/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:30.594587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.692569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.712060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd361562-e20a-4baf-bdc7-122ffa12086d None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.716532 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc6f7-f1e0-4ca9-a870-c95a035eac02 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.716956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc6f7-f1e0-4ca9-a870-c95a035eac02 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.717362 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc6f7-f1e0-4ca9-a870-c95a035eac02 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.719983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 539/1069] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:30.725651 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.726392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.726392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.749709 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:30.751005 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:30.766915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.767690 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.768360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.768360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.769705 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:30.770033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.772635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.798587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 531/1070] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:30.798587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.838256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 540/1071] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:30.860231 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.925779 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475f174-548b-4ba5-8329-1b8f0bd731cd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.976166 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.977442 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.977728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.978311 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.983604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.986914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.988895 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.990081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93b68-2bb6-42bf-a581-01e9accf15f3 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:31.004661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:31.005638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:31.007111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 532/1072] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:31.011579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:31.015838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:31.031298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.032026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.032306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.032583 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.043399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:31.052003 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:31.052987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:31.059022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 541/1073] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:31.059836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:31.124574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a54be-bf37-4da2-8517-630744cbf4a5 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.164001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.164797 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.165203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.165466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.166252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:31.166540 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.167971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.177275 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a4b9d-261f-4a35-8ef8-928d53569f00 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.186103 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 533/1074] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:31.187092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:31.218181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.218181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.218181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.218181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.218633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:31.218633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.219578 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.229857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 542/1075] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:31.235655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:31.257381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:31.270068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:31.318073 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e493b-37a0-4f91-928b-5582515d2489 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.357344 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f4afd-fe9c-41f5-9da6-cc2d83dfcd59 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.359519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.361278 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.361564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.361837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/2135d214a8274e60ac795f71a0697433/roles/75e31c59f5f34919abea5e57fa7d7cab` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.364789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=2135d214a8274e60ac795f71a0697433, role_id=75e31c59f5f34919abea5e57fa7d7cab)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.367095 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.403191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.404074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.404399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.404713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] PATH_INFO: `/v3/users/afc03f3d11114dbbb73d2ca8e53387d5` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.407210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] RBAC: Authorizing `identity:get_user(user_id=afc03f3d11114dbbb73d2ca8e53387d5)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.408795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.411177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 543/1076] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:31 2022] GET /identity/v3/users/afc03f3d11114dbbb73d2ca8e53387d5 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:31.417256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:31.436868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 534/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:31 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/2135d214a8274e60ac795f71a0697433/roles/75e31c59f5f34919abea5e57fa7d7cab => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:31.552616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:31.571399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41377-0164-439f-a413-627d64d1bcec None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.623173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd684a-eecc-4a08-8716-a5c84e2d1daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.624043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd684a-eecc-4a08-8716-a5c84e2d1daa None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.624327 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd684a-eecc-4a08-8716-a5c84e2d1daa None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.624603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd684a-eecc-4a08-8716-a5c84e2d1daa None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.625796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd684a-eecc-4a08-8716-a5c84e2d1daa None admin] RBAC: Authorizing `identity:get_project(project_id=353815900819446194fdb3eb16e81712)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.627315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd684a-eecc-4a08-8716-a5c84e2d1daa None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.629072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 544/1078] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:31 2022] GET /identity/v3/projects/353815900819446194fdb3eb16e81712 => generated 311 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:31.635580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:31.756566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f4824-fae0-4b61-975a-387e217e4a08 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.783439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:31.795887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278abf6c-96cc-4dcf-9e7a-eab9c7751d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.797194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278abf6c-96cc-4dcf-9e7a-eab9c7751d14 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.797194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278abf6c-96cc-4dcf-9e7a-eab9c7751d14 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.797455 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278abf6c-96cc-4dcf-9e7a-eab9c7751d14 None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712/users/afc03f3d11114dbbb73d2ca8e53387d5/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.799475 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278abf6c-96cc-4dcf-9e7a-eab9c7751d14 None admin] RBAC: Authorizing `identity:create_grant(project_id=353815900819446194fdb3eb16e81712, user_id=afc03f3d11114dbbb73d2ca8e53387d5, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.800972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278abf6c-96cc-4dcf-9e7a-eab9c7751d14 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.884516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 535/1079] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:31 2022] PUT /identity/v3/projects/353815900819446194fdb3eb16e81712/users/afc03f3d11114dbbb73d2ca8e53387d5/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:32.289468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c20c-833a-4e48-8753-57d128516ea1 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:32.289988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c20c-833a-4e48-8753-57d128516ea1 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:32.290119 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c20c-833a-4e48-8753-57d128516ea1 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:32.291444 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 545/1080] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:32.296697 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:32.296987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:32.297261 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:32.317339 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:32.318568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:32.389759 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 536/1081] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:32.411746 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491c20c-833a-4e48-8753-57d128516ea1 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:32.515525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491c20c-833a-4e48-8753-57d128516ea1 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:32.557786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:32.558474 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:32.558755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:32.559025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:32.586960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:32.632821 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:32.634341 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:32.731650 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 546/1082] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:32.737376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:32.862634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c40a4-ef21-4b14-ad66-92b7a164b95f None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:32.877139 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:32.896925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:32.897728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:32.899454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:32.899454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:32.908370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:32.910379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f06cab66-5971-44e2-a1cd-0426272e9320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:32.929302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:32.935846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:32.936752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:32.936752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:32.936752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:32.936752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:32.936752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 537/1083] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:32.937376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:32.993989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:33.042237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa13782-c36a-43d5-b906-632efc64ef61 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:33.076136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:33.076804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.077084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.077359 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.078104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:33.078412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:33.079804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaab93-6495-4414-a906-96cb36440558 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:33.097522 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 547/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:33.493038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c416a6-96dc-4712-a79c-b779572e7274 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.493591 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c416a6-96dc-4712-a79c-b779572e7274 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.493591 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c416a6-96dc-4712-a79c-b779572e7274 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.495090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 538/1085] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:33.500265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.500659 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.500814 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.518593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:33.520284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:33.597134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 548/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:33.620372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c416a6-96dc-4712-a79c-b779572e7274 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:33.721987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c416a6-96dc-4712-a79c-b779572e7274 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:33.759920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:33.760748 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.761024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.761289 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.771983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:33.780236 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:33.781387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:33.782297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 539/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:33.787763 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:33.832527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:33.886982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ce55-1fe0-4038-802f-75202bd92109 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:33.920797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:33.921457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.921733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.922002 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.922767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:33.923067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:33.924418 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a46d-b20f-43a1-b70a-ed2351084ed4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:33.937528 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 549/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:33.938712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:33.943593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72132243-6c20-4aba-9292-e7572bdc78bc None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.943888 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72132243-6c20-4aba-9292-e7572bdc78bc None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.944169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72132243-6c20-4aba-9292-e7572bdc78bc None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.945501 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 550/1089] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:33.950437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.950722 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.951000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.967697 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:33.968787 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:33.969990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:34.036168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 551/1090] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:34.048091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac439d7-11b1-42a6-8fe8-b64a4854c737 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.059301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.083723 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.084526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.084809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.085099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.093477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.096252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.097737 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:34.116107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:34.117211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:34.118172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:34.118172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:34.118172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:34.118172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:34.118172 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 540/1091] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:34.123157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.162033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c54dc0-a254-485f-898d-9c9223db70d2 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.195742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.196527 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.196819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.197083 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.200383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.207130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:34.215122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:34.216169 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:34.217007 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 552/1092] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:34.222188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.223252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249fa3-7011-45fc-af59-a628e36d830b None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.257572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.258258 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.258539 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.258810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.259529 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:34.259829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.261408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.273264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 541/1093] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:34.278882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.315745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.321954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d47684-4185-495f-b56b-34d50cc6df98 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.360566 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.361525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.361806 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.362123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.363483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:34.363796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.365172 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.373306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 553/1094] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:34.379048 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.386411 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39564de0-2d9e-4a23-a8dd-02cf48b04ebd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.420064 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.420875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.421158 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.421427 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.429390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.430869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.432712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:34.445675 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:34.451209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:34.452159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 542/1095] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:34.453898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.477021 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd518ec-3aba-4f06-b818-bfbf54bb96b2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.510960 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.511835 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.512125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.512401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] PATH_INFO: `/v3/users/afc03f3d11114dbbb73d2ca8e53387d5` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.513602 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] RBAC: Authorizing `identity:get_user(user_id=afc03f3d11114dbbb73d2ca8e53387d5)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.515068 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.521270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 554/1096] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/users/afc03f3d11114dbbb73d2ca8e53387d5 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:34.522267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.553537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.556656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c48963-c48e-4ea8-b886-4a5267facde7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.593138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.593731 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.594034 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.594309 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.595152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:34.595454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.596882 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.613750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 543/1097] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:34.619727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.631621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d26a8e-5891-455b-a59f-efb832500dc3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.666950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.667600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.667880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.668151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.674282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.716844 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954bce0-51a0-40f5-a18d-fe864b861672 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.735672 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.737119 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.751382 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.752054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.752334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.752599 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.764062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.824694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.826175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.839486 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 555/1098] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:34.845317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.880505 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.930946 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 544/1099] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 18:44:34.936687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.948802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e762c-6ae0-414b-82be-cefbcd9f956a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.973176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.982304 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.983162 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.988725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.988725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.988725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] RBAC: Authorizing `identity:get_project(project_id=353815900819446194fdb3eb16e81712)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.988725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.988725 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 556/1100] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/projects/353815900819446194fdb3eb16e81712 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:34.994744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:35.033289 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c347a73-dec6-4ac7-aa04-5a32afc27a17 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:35.067632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:35.068586 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:35.068869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:35.069147 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:35.079097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:35.080531 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:35.093193 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6aac9-298d-48c2-9e5c-abbbce6e98fd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:35.107223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:35.108418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:35.109328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:35.109328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:35.109328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:35.109328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:35.109328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 545/1101] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:35.109328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:35.116470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:35.137013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:35.137667 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:35.137943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:35.141911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:35.141911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=353815900819446194fdb3eb16e81712, user.id=afc03f3d11114dbbb73d2ca8e53387d5) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:35.141911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:35.141911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:35.160476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 557/1102] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/role_assignments?user.id=afc03f3d11114dbbb73d2ca8e53387d5&scope.project.id=353815900819446194fdb3eb16e81712&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:35.217173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72238dd5-d237-4297-9980-f9c7b92d7878 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:35.256223 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:35.257056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:35.257338 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:35.257610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:35.258387 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:35.258695 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:35.260114 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:35.272888 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 546/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:35.278766 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:35.340672 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:35.383144 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8dd50-0284-4e1a-87cb-6e40dd063e20 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:35.416347 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:35.417141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:35.417419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:35.417691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:35.424246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:35.426348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:35.440408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:35.441453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:44:35.442424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 558/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:35.447543 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:35.512527 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:35.554706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc14f1f-6a84-45ee-ba03-54c8a65a1e8a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:35.595542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:35.596182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:35.596463 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:35.599447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:35.599447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:35.599447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:35.599950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ea835-2987-4190-ac52-6f530bf57bb1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:35.613312 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 547/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:35.619394 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:35.717105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9bf42-8f0a-46a7-9387-444a0a07ca2a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:35.734425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:35.750628 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:35.751280 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:35.751561 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:35.751836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:35.752661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] RBAC: Adding query filter params (role.id=75e31c59f5f34919abea5e57fa7d7cab, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=2135d214a8274e60ac795f71a0697433) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:35.752976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:35.754503 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f067ba-e58f-4ee3-a409-590f654c192c None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:35.760205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 559/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:35 2022] GET /identity/v3/role_assignments?user.id=2135d214a8274e60ac795f71a0697433&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=75e31c59f5f34919abea5e57fa7d7cab => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:37.194769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f9dab-3f86-4c6f-99ca-a5dcaaa7d777 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:37.195300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f9dab-3f86-4c6f-99ca-a5dcaaa7d777 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:37.195300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f9dab-3f86-4c6f-99ca-a5dcaaa7d777 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:37.196761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 548/1107] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:37.202031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:37.202468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:37.202587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:37.220889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:37.222089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:37.289483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 560/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:37.313650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f9dab-3f86-4c6f-99ca-a5dcaaa7d777 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:37.412857 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f9dab-3f86-4c6f-99ca-a5dcaaa7d777 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:37.437905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:37.445953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1377c1f-cb27-4bee-8fa7-0df61024694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:37.446714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377c1f-cb27-4bee-8fa7-0df61024694f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:37.446997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377c1f-cb27-4bee-8fa7-0df61024694f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:37.447270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377c1f-cb27-4bee-8fa7-0df61024694f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:37.448444 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1377c1f-cb27-4bee-8fa7-0df61024694f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:37.449873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1377c1f-cb27-4bee-8fa7-0df61024694f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:37.451506 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 549/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:37.457865 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:37.553975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33498-00b0-4bc0-bfb6-1b17ae6d0fe8 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:37.567263 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:37.588366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a738717-cb0a-4215-ae8d-cb93503d2e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:37.589292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a738717-cb0a-4215-ae8d-cb93503d2e87 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:37.589568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a738717-cb0a-4215-ae8d-cb93503d2e87 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:37.589847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a738717-cb0a-4215-ae8d-cb93503d2e87 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:37.590577 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a738717-cb0a-4215-ae8d-cb93503d2e87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:37.592023 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a738717-cb0a-4215-ae8d-cb93503d2e87 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:37.665278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 561/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:37 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:39.755995 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320eb88-672b-4f0f-8113-eb98e4047406 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:39.756521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320eb88-672b-4f0f-8113-eb98e4047406 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:39.756521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320eb88-672b-4f0f-8113-eb98e4047406 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:39.757920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 550/1111] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:39.763578 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:39.763998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:39.764134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:39.782763 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:39.783984 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:39.852219 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 562/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:39.875850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320eb88-672b-4f0f-8113-eb98e4047406 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:39.979578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320eb88-672b-4f0f-8113-eb98e4047406 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:40.014776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:40.015643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:40.015925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:40.016201 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] PATH_INFO: `/v3/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:40.017716 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] RBAC: Authorizing `identity:get_role(role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:40.019125 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:40.020542 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 551/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:39 2022] GET /identity/v3/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:40.026898 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:40.128084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:40.132427 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b4221-b505-4778-899c-7f69be2855e1 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:40.166409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:40.167290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:40.167571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:40.167844 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] PATH_INFO: `/v3/users/3d9c9fc415cd4bc2804f237662862edc` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:40.181978 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9c9fc415cd4bc2804f237662862edc)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:40.183457 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:40.188333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 563/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:40 2022] GET /identity/v3/users/3d9c9fc415cd4bc2804f237662862edc => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:40.191390 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:40.265564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:40.303227 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1038e93-0eb9-4117-8be4-bc71748968f7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:40.338522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:40.339175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:40.339458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:40.339733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:40.407043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:40.408422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:40.475916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:40.514173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 552/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:40.520445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:40.634136 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31d761-4aef-4dca-bcbc-4aa206f5ea24 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:40.670986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e34c0-2fdd-4e26-a9c5-901dd27149e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:40.671776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e34c0-2fdd-4e26-a9c5-901dd27149e6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:40.672061 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e34c0-2fdd-4e26-a9c5-901dd27149e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:40.672336 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e34c0-2fdd-4e26-a9c5-901dd27149e6 None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:40.673549 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e34c0-2fdd-4e26-a9c5-901dd27149e6 None admin] RBAC: Authorizing `identity:get_project(project_id=353815900819446194fdb3eb16e81712)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:40.675141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e34c0-2fdd-4e26-a9c5-901dd27149e6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:40.677610 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 564/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:40 2022] GET /identity/v3/projects/353815900819446194fdb3eb16e81712 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:40.684028 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:40.786606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c5f7f-f238-4b5e-8973-01a9fac5ec8e None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:40.824456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:40.825107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:40.825385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:40.825653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:40.826516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] RBAC: Adding query filter params (role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=353815900819446194fdb3eb16e81712, user.id=3d9c9fc415cd4bc2804f237662862edc) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:40.826824 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:40.828267 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfb053-6a9e-426a-ba0d-36cf22b0f57c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:40.834741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 553/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:40 2022] GET /identity/v3/role_assignments?user.id=3d9c9fc415cd4bc2804f237662862edc&scope.project.id=353815900819446194fdb3eb16e81712&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:42.865452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efa3a3-5abc-40d2-976b-7cb7b6405c56 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.865956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efa3a3-5abc-40d2-976b-7cb7b6405c56 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.865956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efa3a3-5abc-40d2-976b-7cb7b6405c56 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.867309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 565/1118] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:42.874656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.875360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.875360 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.896786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:42.898028 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:42.921689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:42.966117 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 554/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:42.988313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efa3a3-5abc-40d2-976b-7cb7b6405c56 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:43.014315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:43.096310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efa3a3-5abc-40d2-976b-7cb7b6405c56 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:43.136274 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:43.137045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:43.137320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:43.137587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] PATH_INFO: `/v3/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:43.139043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] RBAC: Authorizing `identity:get_role(role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:43.140397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:43.141779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 566/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:42 2022] GET /identity/v3/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:43.147727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:43.245097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192a4cb-3ac1-404b-90f2-ff0f874b02bb None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:43.284260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:43.285142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:43.285421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:43.285694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] PATH_INFO: `/v3/users/3d9c9fc415cd4bc2804f237662862edc` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:43.286976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] RBAC: Authorizing `identity:get_user(user_id=3d9c9fc415cd4bc2804f237662862edc)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:43.288421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:43.290096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 555/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:43 2022] GET /identity/v3/users/3d9c9fc415cd4bc2804f237662862edc => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:43.295988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:43.397754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:43.406432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54f46d-22e0-42a6-b344-06c6ecdb97cd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:43.446943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820637f-1f02-4dba-857e-b96b0c1ec81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:43.447815 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820637f-1f02-4dba-857e-b96b0c1ec81e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:43.448091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820637f-1f02-4dba-857e-b96b0c1ec81e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:43.448361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820637f-1f02-4dba-857e-b96b0c1ec81e None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:43.449563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820637f-1f02-4dba-857e-b96b0c1ec81e None admin] RBAC: Authorizing `identity:get_project(project_id=353815900819446194fdb3eb16e81712)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:43.451016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820637f-1f02-4dba-857e-b96b0c1ec81e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:43.452737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 567/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:43 2022] GET /identity/v3/projects/353815900819446194fdb3eb16e81712 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:43.459259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:43.486654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:43.563439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396799f9-8e7d-435a-b18b-6bd3e65880b1 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:43.597831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20f79f-90b7-4c48-85dd-3702cbdcc376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:43.598813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f79f-90b7-4c48-85dd-3702cbdcc376 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:43.599077 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f79f-90b7-4c48-85dd-3702cbdcc376 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:43.599348 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f79f-90b7-4c48-85dd-3702cbdcc376 None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712/users/3d9c9fc415cd4bc2804f237662862edc/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:43.601518 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20f79f-90b7-4c48-85dd-3702cbdcc376 None admin] RBAC: Authorizing `identity:create_grant(project_id=353815900819446194fdb3eb16e81712, user_id=3d9c9fc415cd4bc2804f237662862edc, role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:43.602985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20f79f-90b7-4c48-85dd-3702cbdcc376 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:43.689008 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 556/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:43 2022] PUT /identity/v3/projects/353815900819446194fdb3eb16e81712/users/3d9c9fc415cd4bc2804f237662862edc/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:45.675521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb43a-5bbb-4ce7-a131-7078ed14947c None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.676045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb43a-5bbb-4ce7-a131-7078ed14947c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.676166 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb43a-5bbb-4ce7-a131-7078ed14947c None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.677463 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 568/1124] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:45.685171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.685637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.685787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.704151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:45.705344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:45.785363 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 557/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:45.809478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb43a-5bbb-4ce7-a131-7078ed14947c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.911229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb43a-5bbb-4ce7-a131-7078ed14947c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:45.951673 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:45.952533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.952811 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.953082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] PATH_INFO: `/v3/roles/d18dacf2bd0a41b29a8401da7ebc71ce` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.954600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] RBAC: Authorizing `identity:get_role(role_id=d18dacf2bd0a41b29a8401da7ebc71ce)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:45.955955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:45.957352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 569/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:45 2022] GET /identity/v3/roles/d18dacf2bd0a41b29a8401da7ebc71ce => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:45.964148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.982024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:46.067114 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eadebc-e45c-4f60-aeef-51920775fc65 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:46.104432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:46.105323 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:46.105601 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:46.105873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] PATH_INFO: `/v3/users/3d9c9fc415cd4bc2804f237662862edc` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:46.112348 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] RBAC: Authorizing `identity:get_user(user_id=3d9c9fc415cd4bc2804f237662862edc)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:46.112348 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:46.112348 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 558/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:45 2022] GET /identity/v3/users/3d9c9fc415cd4bc2804f237662862edc => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:46.116558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:46.148132 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:46.221436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431294b8-8bf4-420e-9ea6-abd457bc23a6 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:46.257829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:46.258511 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:46.258796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:46.259073 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:46.326636 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:46.327990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:46.356708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:46.427423 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 570/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:46.433532 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:46.528430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:46.545024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dc946-6dee-4c60-9e78-426c64025fab None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:46.580371 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbdfc2-0132-4ecb-9f61-ba3890825c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:46.581149 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbdfc2-0132-4ecb-9f61-ba3890825c93 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:46.581435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbdfc2-0132-4ecb-9f61-ba3890825c93 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:46.581714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbdfc2-0132-4ecb-9f61-ba3890825c93 None admin] PATH_INFO: `/v3/projects/353815900819446194fdb3eb16e81712` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:46.582976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbdfc2-0132-4ecb-9f61-ba3890825c93 None admin] RBAC: Authorizing `identity:get_project(project_id=353815900819446194fdb3eb16e81712)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:46.585058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbdfc2-0132-4ecb-9f61-ba3890825c93 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:46.587097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 559/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:46 2022] GET /identity/v3/projects/353815900819446194fdb3eb16e81712 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:46.593625 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:46.696500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0ce5-1d99-4c7d-b153-9521e8450d9d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:46.723380 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:46.731579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:46.732244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:46.732526 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:46.732803 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:46.733642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] RBAC: Adding query filter params (role.id=d18dacf2bd0a41b29a8401da7ebc71ce, scope.project.id=353815900819446194fdb3eb16e81712, user.id=3d9c9fc415cd4bc2804f237662862edc) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:46.733957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:46.735416 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9771b0-9033-45a1-84b2-ccb4d61cc795 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:46.742557 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 571/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:46 2022] GET /identity/v3/role_assignments?user.id=3d9c9fc415cd4bc2804f237662862edc&scope.project.id=353815900819446194fdb3eb16e81712&role.id=d18dacf2bd0a41b29a8401da7ebc71ce => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:48.820983 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2d89-c914-4f8a-a4b6-5a9e91d73729 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.821516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2d89-c914-4f8a-a4b6-5a9e91d73729 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.821516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2d89-c914-4f8a-a4b6-5a9e91d73729 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.822864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 560/1131] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:48.828256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.828689 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.828802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.848151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:48.849901 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:48.918653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 572/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:48.942443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2d89-c914-4f8a-a4b6-5a9e91d73729 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:49.048988 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2d89-c914-4f8a-a4b6-5a9e91d73729 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:49.087058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1e763-1b96-4b5f-8ee3-2931f188cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:49.087787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1e763-1b96-4b5f-8ee3-2931f188cc3f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:49.088070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1e763-1b96-4b5f-8ee3-2931f188cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:49.088343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1e763-1b96-4b5f-8ee3-2931f188cc3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:49.089536 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1e763-1b96-4b5f-8ee3-2931f188cc3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:49.091016 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1e763-1b96-4b5f-8ee3-2931f188cc3f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:49.092662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 561/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:49.098669 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:49.207677 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21734ba0-2b0f-4799-8fb2-a8ecc8b2f6e5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:49.232310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:49.244542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa35c5a-d0cf-4d78-b205-7ea5c77aecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:49.245290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa35c5a-d0cf-4d78-b205-7ea5c77aecf8 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:49.245575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa35c5a-d0cf-4d78-b205-7ea5c77aecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:49.245853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa35c5a-d0cf-4d78-b205-7ea5c77aecf8 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:49.250883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa35c5a-d0cf-4d78-b205-7ea5c77aecf8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:49.250883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa35c5a-d0cf-4d78-b205-7ea5c77aecf8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:49.354508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 573/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:49 2022] POST /identity/v3/projects => generated 311 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:51.449096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7f651-55da-4c3d-a792-4d142e8ccf19 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.449656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7f651-55da-4c3d-a792-4d142e8ccf19 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.449656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7f651-55da-4c3d-a792-4d142e8ccf19 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.451043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 562/1135] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:51.456371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.456794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.456903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.474852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:51.476037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:51.549605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 574/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:51.574783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7f651-55da-4c3d-a792-4d142e8ccf19 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:51.596823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:51.681893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7f651-55da-4c3d-a792-4d142e8ccf19 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:51.716727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b100-d950-4f75-9a7e-8c08c0574415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:51.717445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b100-d950-4f75-9a7e-8c08c0574415 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.717724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b100-d950-4f75-9a7e-8c08c0574415 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.717997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b100-d950-4f75-9a7e-8c08c0574415 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.719215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b100-d950-4f75-9a7e-8c08c0574415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:51.720657 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b100-d950-4f75-9a7e-8c08c0574415 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:51.722326 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 563/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:51.728754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:51.829428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b657e5-36f2-40e2-a8e7-9dfb7441fdd5 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:51.831998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:51.865353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930e48b-b298-4cd1-9ff6-e2627eb18917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:51.866045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930e48b-b298-4cd1-9ff6-e2627eb18917 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.866369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930e48b-b298-4cd1-9ff6-e2627eb18917 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.867194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930e48b-b298-4cd1-9ff6-e2627eb18917 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.867907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930e48b-b298-4cd1-9ff6-e2627eb18917 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:51.869330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930e48b-b298-4cd1-9ff6-e2627eb18917 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:51.947331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 575/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:51 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:54.015948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f34c5-0e32-4065-bfef-83fb25d5d5b5 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.016470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f34c5-0e32-4065-bfef-83fb25d5d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.016470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f34c5-0e32-4065-bfef-83fb25d5d5b5 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.017863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 564/1139] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:54.023770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.024251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.024345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.042499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:54.043705 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:54.111710 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 576/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:54.134874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f34c5-0e32-4065-bfef-83fb25d5d5b5 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:54.225750 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:54.238295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f34c5-0e32-4065-bfef-83fb25d5d5b5 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:54.274902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:54.275608 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.275893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.276168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.286535 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:54.294874 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:54.296010 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:54.296868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 565/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:54.302475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:54.393964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:54.405848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d218504-c915-4ba0-b5fb-cdab3aa14375 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:54.440739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:54.441499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.441783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.442084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.442837 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:54.443143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:54.444513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:54.453211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 577/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:54.459226 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:54.560767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ae27f-4116-4c56-9c2b-e821b12ab5e6 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:54.595445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:54.596343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.596609 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.596887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] PATH_INFO: `/v3/users/741deb05cfaa4fb187aa8b88b40d9aac` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.611464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] RBAC: Authorizing `identity:get_user(user_id=741deb05cfaa4fb187aa8b88b40d9aac)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:54.612936 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:54.614784 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 566/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:54 2022] GET /identity/v3/users/741deb05cfaa4fb187aa8b88b40d9aac => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:54.620819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:54.727583 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794e063-0565-486c-a03c-a809f87b5b18 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:54.761185 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:54.761828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.762134 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.762408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.784735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:54.832243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:54.833599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:54.947702 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 578/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 18:44:54.955405 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:54.966204 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:55.094991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7db7-e4be-4b11-b68f-a6700c1e6559 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:55.131514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904340b2-4434-425f-ac8f-2ae94916d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:55.132377 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904340b2-4434-425f-ac8f-2ae94916d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:55.133254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904340b2-4434-425f-ac8f-2ae94916d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:55.133560 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904340b2-4434-425f-ac8f-2ae94916d6a0 None admin] PATH_INFO: `/v3/projects/ad4f7b7deee8466093b6f4a44973d63f` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:55.134776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904340b2-4434-425f-ac8f-2ae94916d6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ad4f7b7deee8466093b6f4a44973d63f)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:55.136255 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904340b2-4434-425f-ac8f-2ae94916d6a0 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:55.137980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 567/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:54 2022] GET /identity/v3/projects/ad4f7b7deee8466093b6f4a44973d63f => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:55.144175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:55.197277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:55.244868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bb782-02ce-405e-b7ff-d3883123ec68 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:55.278177 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:55.278827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:55.279108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:55.279376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:55.280202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=ad4f7b7deee8466093b6f4a44973d63f, user.id=741deb05cfaa4fb187aa8b88b40d9aac) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:55.280506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:55.281933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd36ec-db4f-46fe-a5c6-6dbc51c5de98 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:55.287568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 579/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:55 2022] GET /identity/v3/role_assignments?user.id=741deb05cfaa4fb187aa8b88b40d9aac&scope.project.id=ad4f7b7deee8466093b6f4a44973d63f&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:57.284074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1f0c4-44b2-4311-9135-efcf93a5559f None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.284610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1f0c4-44b2-4311-9135-efcf93a5559f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.284610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1f0c4-44b2-4311-9135-efcf93a5559f None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.285998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 568/1147] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:57.291582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.292032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.292122 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.310523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:57.311716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:57.388074 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 580/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:57.409775 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1f0c4-44b2-4311-9135-efcf93a5559f None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:57.490179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:57.518848 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1f0c4-44b2-4311-9135-efcf93a5559f None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:57.555107 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:57.556423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.556702 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.556972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.567294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:57.575026 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:57.576229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:44:57.577277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 569/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:57.582366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:57.630028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:57.683032 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd466-abfc-49c7-bfad-1e2f115ca97e None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:57.719853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:57.720515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.720796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.721070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.721822 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:57.722157 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:57.723523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:57.732991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 581/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:57.739175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:57.841757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e68534-b9c2-430d-8f94-c52831d84d60 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:57.875280 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:57.876158 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.876439 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.876717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] PATH_INFO: `/v3/users/741deb05cfaa4fb187aa8b88b40d9aac` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.877928 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] RBAC: Authorizing `identity:get_user(user_id=741deb05cfaa4fb187aa8b88b40d9aac)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:57.879394 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:57.881050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 570/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:57 2022] GET /identity/v3/users/741deb05cfaa4fb187aa8b88b40d9aac => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:57.887374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:57.998090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69481df9-2ae7-41eb-aa44-7234d88091c5 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:58.007517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:58.032366 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773bb5b-34fd-4ba5-bc9b-fe1d258018fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:58.033222 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773bb5b-34fd-4ba5-bc9b-fe1d258018fe None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:58.033507 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773bb5b-34fd-4ba5-bc9b-fe1d258018fe None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:58.033781 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773bb5b-34fd-4ba5-bc9b-fe1d258018fe None admin] PATH_INFO: `/v3/projects/ad4f7b7deee8466093b6f4a44973d63f` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:58.035085 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773bb5b-34fd-4ba5-bc9b-fe1d258018fe None admin] RBAC: Authorizing `identity:get_project(project_id=ad4f7b7deee8466093b6f4a44973d63f)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:58.036507 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773bb5b-34fd-4ba5-bc9b-fe1d258018fe None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:58.038259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 582/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:57 2022] GET /identity/v3/projects/ad4f7b7deee8466093b6f4a44973d63f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:58.044936 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:58.047924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:58.166742 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c847f9-209e-4cd1-b219-9742039fccec None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:58.200982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d04b5d-67e5-44fc-85e3-d9c68a8995c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:58.201879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d04b5d-67e5-44fc-85e3-d9c68a8995c9 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:58.202192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d04b5d-67e5-44fc-85e3-d9c68a8995c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:58.202469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d04b5d-67e5-44fc-85e3-d9c68a8995c9 None admin] PATH_INFO: `/v3/projects/ad4f7b7deee8466093b6f4a44973d63f/users/741deb05cfaa4fb187aa8b88b40d9aac/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:58.204458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d04b5d-67e5-44fc-85e3-d9c68a8995c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4f7b7deee8466093b6f4a44973d63f, user_id=741deb05cfaa4fb187aa8b88b40d9aac, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:58.205908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d04b5d-67e5-44fc-85e3-d9c68a8995c9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:58.303548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 571/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:58 2022] PUT /identity/v3/projects/ad4f7b7deee8466093b6f4a44973d63f/users/741deb05cfaa4fb187aa8b88b40d9aac/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:45:00.303717 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f43be-0e33-4900-9d4c-67b8c7a3b0a9 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:00.304298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f43be-0e33-4900-9d4c-67b8c7a3b0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:00.304298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f43be-0e33-4900-9d4c-67b8c7a3b0a9 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:00.306155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 583/1154] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:00.311614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:00.312091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:00.312191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:00.330996 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:00.333038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:00.391046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:00.402588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 572/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:00.425272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f43be-0e33-4900-9d4c-67b8c7a3b0a9 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:00.529328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f43be-0e33-4900-9d4c-67b8c7a3b0a9 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:00.565194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:00.565907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:00.566215 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:00.566489 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:00.577762 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:00.585585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:00.587236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:45:00.588606 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 584/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:00.593970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:00.692810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d33185-47cb-4a19-9255-38a0939864fe None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:00.730030 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:00.730701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:00.730981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:00.731251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:00.732001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:00.732304 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:00.733672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:00.742055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 573/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:00.750887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:00.786487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:00.861558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94880-7f8e-4194-b084-c0af82198b87 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:00.897290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:00.898192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:00.898475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:00.898748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] PATH_INFO: `/v3/users/741deb05cfaa4fb187aa8b88b40d9aac` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:00.900503 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] RBAC: Authorizing `identity:get_user(user_id=741deb05cfaa4fb187aa8b88b40d9aac)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:00.901930 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:00.903623 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 585/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:00 2022] GET /identity/v3/users/741deb05cfaa4fb187aa8b88b40d9aac => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:00.909458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:00.969810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:01.014654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970791d8-1497-476e-9715-e7973a885ccd None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:01.047503 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:01.048150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:01.048429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:01.048700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:01.121205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:01.123219 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:01.192301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:01.237839 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 574/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:45:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:45:01.244292 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:01.347682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6be3-6e79-4f6c-a303-3d1f76d5bc66 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:01.350080 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:01.383079 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0e6d-1b0e-4cf5-843c-c25009e8effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:01.384178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0e6d-1b0e-4cf5-843c-c25009e8effa None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:01.384464 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0e6d-1b0e-4cf5-843c-c25009e8effa None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:01.384742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0e6d-1b0e-4cf5-843c-c25009e8effa None admin] PATH_INFO: `/v3/projects/ad4f7b7deee8466093b6f4a44973d63f` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:01.385940 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0e6d-1b0e-4cf5-843c-c25009e8effa None admin] RBAC: Authorizing `identity:get_project(project_id=ad4f7b7deee8466093b6f4a44973d63f)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:01.387404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0e6d-1b0e-4cf5-843c-c25009e8effa None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:01.389143 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 586/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:01 2022] GET /identity/v3/projects/ad4f7b7deee8466093b6f4a44973d63f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:01.395343 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:01.518330 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf498d-6d08-406f-ac8c-bbd1ca0ec95a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:01.553341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:01.554022 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:01.554313 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:01.554588 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:01.556013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=ad4f7b7deee8466093b6f4a44973d63f, user.id=741deb05cfaa4fb187aa8b88b40d9aac) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:01.556324 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:01.557764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101a172-2efc-4935-8cb1-96a4c2fbc5a7 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:01.563821 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 575/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:45:01 2022] GET /identity/v3/role_assignments?user.id=741deb05cfaa4fb187aa8b88b40d9aac&scope.project.id=ad4f7b7deee8466093b6f4a44973d63f&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:03.638510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09553b-7e09-4c4d-9272-3e21d50ba916 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:03.639060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09553b-7e09-4c4d-9272-3e21d50ba916 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:03.639060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09553b-7e09-4c4d-9272-3e21d50ba916 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:03.640441 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 587/1162] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:03.646243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:03.646807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:03.646807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:03.665389 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:03.666617 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:03.680774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:03.734975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 576/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:03.757026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09553b-7e09-4c4d-9272-3e21d50ba916 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:03.859131 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09553b-7e09-4c4d-9272-3e21d50ba916 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:03.892670 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:03.893317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:03.893600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:03.893871 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:03.934843 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:03.960743 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:03.962254 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:04.061163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 588/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:45:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:45:04.067462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:04.164793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45712189-dd63-4b47-a44d-491562aa6af4 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:04.197359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:04.198178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:04.198457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:04.198727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:04.205222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:04.206678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-47a17956-94a4-4212-9eef-7679abdddad2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:04.221372 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:04.222630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:45:04.223512 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 577/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:45:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:04.229227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:04.297258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:04.328258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3beaa-c586-4fbb-8205-1fd11205a1d3 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:04.364823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:04.365471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:04.365750 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:04.366038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:04.366761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:04.367057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:04.368424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09a7a-70bc-46cb-adfa-f262ac80b726 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:04.375578 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 589/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 25 18:45:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:06.435986 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bfc53-88ee-46f1-9097-673b058c673e None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:06.436545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bfc53-88ee-46f1-9097-673b058c673e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:06.436545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bfc53-88ee-46f1-9097-673b058c673e None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:06.437891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 578/1167] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:06.443076 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697cef-2f8c-4617-b960-119a414f17a7 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:06.443477 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697cef-2f8c-4617-b960-119a414f17a7 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:06.443638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697cef-2f8c-4617-b960-119a414f17a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:06.462121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-3d697cef-2f8c-4617-b960-119a414f17a7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:06.463371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-3d697cef-2f8c-4617-b960-119a414f17a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:06.533474 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 590/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:06.559846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bfc53-88ee-46f1-9097-673b058c673e None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:06.572936 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:06.663828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bfc53-88ee-46f1-9097-673b058c673e None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:06.695932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320d896-0311-4aee-88c0-23ce2b34b57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:06.696511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320d896-0311-4aee-88c0-23ce2b34b57c None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:06.697328 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320d896-0311-4aee-88c0-23ce2b34b57c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:06.697600 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320d896-0311-4aee-88c0-23ce2b34b57c None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:06.698374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320d896-0311-4aee-88c0-23ce2b34b57c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:06.699873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320d896-0311-4aee-88c0-23ce2b34b57c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:06.773460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 579/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Mon Jul 25 18:45:06 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:45:08.814908 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7becfe8-f823-463d-9d95-c0086d699e47 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:08.815421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7becfe8-f823-463d-9d95-c0086d699e47 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:08.815421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7becfe8-f823-463d-9d95-c0086d699e47 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:08.816732 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 591/1170] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:08.822769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:08.823192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:08.823334 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:08.843237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:08.844424 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:08.918260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:08.924089 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 580/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:08.950381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7becfe8-f823-463d-9d95-c0086d699e47 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:09.023181 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:09.064568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7becfe8-f823-463d-9d95-c0086d699e47 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:09.103252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1bc31-acf0-47bb-b8b9-4bcc4daee818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:09.104025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bc31-acf0-47bb-b8b9-4bcc4daee818 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:09.104304 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bc31-acf0-47bb-b8b9-4bcc4daee818 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:09.104574 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bc31-acf0-47bb-b8b9-4bcc4daee818 None admin] PATH_INFO: `/v3/domains/850f3ec6b76b4f04af04b6270a11b971` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:09.116168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1bc31-acf0-47bb-b8b9-4bcc4daee818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=850f3ec6b76b4f04af04b6270a11b971)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:09.117616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1bc31-acf0-47bb-b8b9-4bcc4daee818 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:09.119415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 592/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Jul 25 18:45:08 2022] GET /identity/v3/domains/850f3ec6b76b4f04af04b6270a11b971 => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:09.125364 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:09.235312 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025351d-b323-41a7-b483-8578b7d144b7 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:09.269537 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b61a2-7d15-4891-9174-93be7488ba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:09.270229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b61a2-7d15-4891-9174-93be7488ba95 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:09.270515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b61a2-7d15-4891-9174-93be7488ba95 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:09.270789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b61a2-7d15-4891-9174-93be7488ba95 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:09.271488 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b61a2-7d15-4891-9174-93be7488ba95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:09.272931 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b61a2-7d15-4891-9174-93be7488ba95 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:09.374183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 581/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:45:09 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:45:11.457140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada62bb-355b-4783-ba91-5b7270e1e5af None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:11.457700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada62bb-355b-4783-ba91-5b7270e1e5af None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:11.457700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada62bb-355b-4783-ba91-5b7270e1e5af None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:11.459109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 593/1174] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:11.464369 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f14791-a241-498a-994d-7431d2187e34 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:11.465101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f14791-a241-498a-994d-7431d2187e34 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:11.465208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f14791-a241-498a-994d-7431d2187e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:11.482887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-99f14791-a241-498a-994d-7431d2187e34 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:11.484064 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-99f14791-a241-498a-994d-7431d2187e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:11.529398 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:11.556251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 582/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:11.581937 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada62bb-355b-4783-ba91-5b7270e1e5af None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:11.687289 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada62bb-355b-4783-ba91-5b7270e1e5af None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:11.713564 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:11.722285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd211d-08bc-44b5-ae29-07a2c4b633ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:11.731277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd211d-08bc-44b5-ae29-07a2c4b633ac None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:11.731277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd211d-08bc-44b5-ae29-07a2c4b633ac None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:11.731277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd211d-08bc-44b5-ae29-07a2c4b633ac None admin] PATH_INFO: `/v3/domains/850f3ec6b76b4f04af04b6270a11b971` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:11.731277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd211d-08bc-44b5-ae29-07a2c4b633ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=850f3ec6b76b4f04af04b6270a11b971)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:11.731277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd211d-08bc-44b5-ae29-07a2c4b633ac None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:11.731277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 594/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Jul 25 18:45:11 2022] GET /identity/v3/domains/850f3ec6b76b4f04af04b6270a11b971 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:11.734670 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f14791-a241-498a-994d-7431d2187e34 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:11.832629 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f14791-a241-498a-994d-7431d2187e34 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:11.866663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2836737-5215-47ce-bee5-764250e52eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:11.867342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2836737-5215-47ce-bee5-764250e52eb4 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:11.867624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2836737-5215-47ce-bee5-764250e52eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:11.867899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2836737-5215-47ce-bee5-764250e52eb4 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:11.868619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2836737-5215-47ce-bee5-764250e52eb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:11.870122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2836737-5215-47ce-bee5-764250e52eb4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:11.968456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 583/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:45:11 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:45:14.008836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880cff6-cada-440a-93ac-bc3d7f2711b6 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:14.009384 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880cff6-cada-440a-93ac-bc3d7f2711b6 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:14.009384 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880cff6-cada-440a-93ac-bc3d7f2711b6 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:14.010868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 595/1178] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:14.016019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:14.016418 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:14.016577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:14.034349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:14.035535 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:14.064445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:14.105317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 584/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:14.128639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880cff6-cada-440a-93ac-bc3d7f2711b6 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:14.227837 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880cff6-cada-440a-93ac-bc3d7f2711b6 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:14.261949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:14.262703 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:14.262986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:14.263260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:14.273256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:14.281482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:14.282516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:45:14.283368 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 596/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:14.288785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:14.384395 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c6d03-2689-4452-833c-14e02759dabe None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:14.417960 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:14.418657 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:14.418944 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:14.419222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:14.419976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:14.420283 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:14.421653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:14.431754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 585/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:14.437724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:14.495156 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:14.538369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec752f-8704-4359-84b2-1a95211a0c3a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:14.571378 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:14.572243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:14.572521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:14.572836 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] PATH_INFO: `/v3/users/e84bcfc2fa72414d8fbca3b39e785714` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:14.585593 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] RBAC: Authorizing `identity:get_user(user_id=e84bcfc2fa72414d8fbca3b39e785714)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:14.587054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:14.589245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 597/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:14 2022] GET /identity/v3/users/e84bcfc2fa72414d8fbca3b39e785714 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:14.595412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:14.624550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:14.697423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a290fde-c9d7-466f-9bf9-2afcb5fc64a8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:14.732385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:14.733049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:14.733467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:14.733720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:14.802487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:14.803868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:14.831130 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:14.919041 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 586/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:45:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:45:14.925047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:15.023108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a64857-cf49-496a-95a0-441d14000eae None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:15.035925 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:15.056464 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ea7b-7be8-4fbb-a39a-1adefb49c6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:15.057785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ea7b-7be8-4fbb-a39a-1adefb49c6d5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:15.058277 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ea7b-7be8-4fbb-a39a-1adefb49c6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:15.058779 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ea7b-7be8-4fbb-a39a-1adefb49c6d5 None admin] PATH_INFO: `/v3/projects/097142633f3e4134a67417fda5b986c6` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:15.060267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33ea7b-7be8-4fbb-a39a-1adefb49c6d5 None admin] RBAC: Authorizing `identity:get_project(project_id=097142633f3e4134a67417fda5b986c6)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:15.061690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33ea7b-7be8-4fbb-a39a-1adefb49c6d5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:15.063452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 598/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:14 2022] GET /identity/v3/projects/097142633f3e4134a67417fda5b986c6 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:15.069773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:15.167369 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e225a-8a95-4425-a97e-48756aac909c None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:15.193859 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:15.202059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:15.202717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:15.202997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:15.203268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:15.204096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=097142633f3e4134a67417fda5b986c6, user.id=e84bcfc2fa72414d8fbca3b39e785714) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:15.204402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:15.205826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4037b87-0444-46d4-8add-3c232770da48 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:15.211202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 587/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:45:15 2022] GET /identity/v3/role_assignments?user.id=e84bcfc2fa72414d8fbca3b39e785714&scope.project.id=097142633f3e4134a67417fda5b986c6&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:17.199614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302573-85a4-4d56-84bb-282d9c7a6d11 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:17.200254 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302573-85a4-4d56-84bb-282d9c7a6d11 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:17.200254 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302573-85a4-4d56-84bb-282d9c7a6d11 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:17.201533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 599/1186] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:17.206634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:17.207066 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:17.207168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:17.225706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:17.226920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:17.290736 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 588/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:17.313661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302573-85a4-4d56-84bb-282d9c7a6d11 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:17.420118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302573-85a4-4d56-84bb-282d9c7a6d11 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:17.456664 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:17.457378 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:17.457660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:17.457934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:17.468662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:17.476346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:17.477523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:45:17.478671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 600/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:17.483831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:17.586262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f950f-c139-46b5-a0c5-4f84fc67a686 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:17.608513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:17.619655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:17.620827 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:17.621101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:17.621367 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:17.622150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:17.622452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:17.623869 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:17.632081 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 589/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:17.638634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:17.680604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:17.740026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307a7e5-4505-45f0-bafb-198292235ec7 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:17.774247 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:17.775118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:17.775401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:17.775675 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] PATH_INFO: `/v3/users/e84bcfc2fa72414d8fbca3b39e785714` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:17.776880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] RBAC: Authorizing `identity:get_user(user_id=e84bcfc2fa72414d8fbca3b39e785714)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:17.778331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:17.779982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 601/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:17 2022] GET /identity/v3/users/e84bcfc2fa72414d8fbca3b39e785714 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:17.785803 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:17.884424 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca08e2b-c163-4d42-8330-43b2a6f6fc28 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:17.917559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71f4ae-b06c-4270-87e6-b01521aae05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:17.918466 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71f4ae-b06c-4270-87e6-b01521aae05f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:17.918735 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71f4ae-b06c-4270-87e6-b01521aae05f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:17.919003 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71f4ae-b06c-4270-87e6-b01521aae05f None admin] PATH_INFO: `/v3/projects/097142633f3e4134a67417fda5b986c6` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:17.920174 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71f4ae-b06c-4270-87e6-b01521aae05f None admin] RBAC: Authorizing `identity:get_project(project_id=097142633f3e4134a67417fda5b986c6)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:17.921592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71f4ae-b06c-4270-87e6-b01521aae05f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:17.923321 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 590/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:17 2022] GET /identity/v3/projects/097142633f3e4134a67417fda5b986c6 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:17.929382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:18.030386 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972eb33-725f-4fe8-b9b1-db045bf1432b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:18.039002 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:18.063307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853475d1-d5d9-463a-9aff-a5e967ab45fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:18.064216 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853475d1-d5d9-463a-9aff-a5e967ab45fd None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:18.066153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853475d1-d5d9-463a-9aff-a5e967ab45fd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:18.066153 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853475d1-d5d9-463a-9aff-a5e967ab45fd None admin] PATH_INFO: `/v3/projects/097142633f3e4134a67417fda5b986c6/users/e84bcfc2fa72414d8fbca3b39e785714/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:18.066827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853475d1-d5d9-463a-9aff-a5e967ab45fd None admin] RBAC: Authorizing `identity:create_grant(project_id=097142633f3e4134a67417fda5b986c6, user_id=e84bcfc2fa72414d8fbca3b39e785714, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:18.068271 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853475d1-d5d9-463a-9aff-a5e967ab45fd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:18.144626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 602/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:45:17 2022] PUT /identity/v3/projects/097142633f3e4134a67417fda5b986c6/users/e84bcfc2fa72414d8fbca3b39e785714/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:45:20.138445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd5298-aa0f-4beb-9002-e4e2fefdd1d6 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:20.138998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd5298-aa0f-4beb-9002-e4e2fefdd1d6 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:20.138998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd5298-aa0f-4beb-9002-e4e2fefdd1d6 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:20.140374 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 591/1193] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:20.145523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:20.145958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:20.146095 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:20.164209 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:20.165375 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:20.230251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 603/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:20.253109 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd5298-aa0f-4beb-9002-e4e2fefdd1d6 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:20.306104 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:20.353919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd5298-aa0f-4beb-9002-e4e2fefdd1d6 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:20.387164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:20.387956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:20.388238 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:20.388572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:20.398926 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:20.407086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:20.408082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:45:20.409069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 592/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:20.414464 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:20.515479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:20.519236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44e5cb-e263-4564-bb2f-a607b2d0cf85 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:20.554625 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:20.555361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:20.555625 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:20.555899 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:20.556642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:20.556943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:20.558872 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:20.567553 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 604/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:20.573384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:20.673905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31527c-5d75-44a0-a60a-c19481a67422 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:20.687059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:20.707102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:20.708060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:20.708345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:20.708619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] PATH_INFO: `/v3/users/e84bcfc2fa72414d8fbca3b39e785714` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:20.709825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] RBAC: Authorizing `identity:get_user(user_id=e84bcfc2fa72414d8fbca3b39e785714)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:20.711745 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:20.713638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 593/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:20 2022] GET /identity/v3/users/e84bcfc2fa72414d8fbca3b39e785714 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:20.720717 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:20.825940 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab89b27-90f2-45c2-9cb8-e6750f4a756b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:20.858637 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:20.859278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:20.859562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:20.859841 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:20.886735 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:20.926361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:20.927723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:21.022307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 605/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:45:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:45:21.028732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:21.146176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f5ee3-ce8e-4528-aaf8-81b0944d9ade None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:21.179549 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846202a-67b0-4ed2-9c56-0c1226a007cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:21.180335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846202a-67b0-4ed2-9c56-0c1226a007cd None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:21.180612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846202a-67b0-4ed2-9c56-0c1226a007cd None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:21.180877 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846202a-67b0-4ed2-9c56-0c1226a007cd None admin] PATH_INFO: `/v3/projects/097142633f3e4134a67417fda5b986c6` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:21.182053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846202a-67b0-4ed2-9c56-0c1226a007cd None admin] RBAC: Authorizing `identity:get_project(project_id=097142633f3e4134a67417fda5b986c6)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:21.183511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846202a-67b0-4ed2-9c56-0c1226a007cd None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:21.185176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 594/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:21 2022] GET /identity/v3/projects/097142633f3e4134a67417fda5b986c6 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:21.191330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:21.257105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:21.297658 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed539ae-7c93-4959-89b6-d6faef319a5e None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:21.329605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:21.330270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:21.330549 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:21.330823 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:21.331643 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=097142633f3e4134a67417fda5b986c6, user.id=e84bcfc2fa72414d8fbca3b39e785714) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:21.331949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:21.333355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701d61b-1340-469a-bece-9eacef30b9e5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:21.339025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 606/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:45:21 2022] GET /identity/v3/role_assignments?user.id=e84bcfc2fa72414d8fbca3b39e785714&scope.project.id=097142633f3e4134a67417fda5b986c6&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:31.175356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=76 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:31.336408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=38 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:35.770764 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eadc6-ba49-4a5d-b7a7-a8b3f171f6f1 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:35.771385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eadc6-ba49-4a5d-b7a7-a8b3f171f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:35.771385 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eadc6-ba49-4a5d-b7a7-a8b3f171f6f1 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:35.772727 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 595/1201] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:35.778453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:35.778847 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:35.779004 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:35.797374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:35.798525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:35.869938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 607/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:35.895373 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eadc6-ba49-4a5d-b7a7-a8b3f171f6f1 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:35.997678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eadc6-ba49-4a5d-b7a7-a8b3f171f6f1 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:36.030762 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:36.031545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:36.031825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:36.032094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:36.042617 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:36.044060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:36.061470 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:36.062694 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:36.063911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:36.063911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:36.063911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:36.063911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:36.063911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 596/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:36.068755 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:36.170466 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea6aa5-0ab2-4cbe-a8bf-92cbc8de4cca None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:36.200027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:36.205704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:36.206393 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:36.206673 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:36.206945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:36.207660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:36.207958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:36.209310 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef3246-4e5a-41c3-8402-b3a80bc8f048 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:36.221282 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 608/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:36.230126 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:36.282791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:36.330402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84505b91-32f3-4383-b264-e16d2137146f None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:36.364613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc9ce7-e05d-4290-9b43-066d22d772d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:36.365351 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc9ce7-e05d-4290-9b43-066d22d772d2 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:36.365635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc9ce7-e05d-4290-9b43-066d22d772d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:36.365911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc9ce7-e05d-4290-9b43-066d22d772d2 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:36.366749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc9ce7-e05d-4290-9b43-066d22d772d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:36.368732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc9ce7-e05d-4290-9b43-066d22d772d2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:36.603159 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 597/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 25 18:46:36 2022] POST /identity/v3/users => generated 251 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:46:38.658863 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7709d8-098f-44b3-9bae-1e646993b091 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.659424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7709d8-098f-44b3-9bae-1e646993b091 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.659424 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7709d8-098f-44b3-9bae-1e646993b091 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.660795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 609/1206] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:38.666283 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.666724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.666831 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.685592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:38.686922 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:38.755476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 598/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:38.779711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7709d8-098f-44b3-9bae-1e646993b091 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.884121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7709d8-098f-44b3-9bae-1e646993b091 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:38.918599 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:38.920005 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.920430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.920568 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.930562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:38.939125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:38.940721 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:38.941783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 610/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:38.946507 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.990208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.049432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986111d-3ae1-4c68-8d41-05a591cd5bb2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.083620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.084296 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.084577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.084851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.085602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:39.085905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.087322 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.095985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 599/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:46:39.102088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.153802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.204417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4823c64-4112-4f20-8f90-c88f0269dfc2 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.238506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.239297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.239581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.239858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.251306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.252767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:39.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:39.271970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:39.272862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:39.272862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:39.272862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:39.272862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:39.272862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 611/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:39.277808 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.375910 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.378740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4627cc-5649-4c14-881e-4167fdc1ad27 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.414062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.414062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.414062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.414062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.414515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:39.414759 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.416300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.429129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 600/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:39.435151 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.540320 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba28a7-3d38-4151-849b-976a07295c51 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.541949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.577603 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.578473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.578741 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.579014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.586235 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.587732 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:39.603849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:39.604965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:39.606249 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 612/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:39.611838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.710509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9eef9-fab4-4f7d-ab41-09d5e8f2a512 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.745766 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.746479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.746761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.747033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.747914 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:39.748232 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.749650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.767402 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 601/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:39.773149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.872466 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3eebd5-428a-4767-ab4b-716b7a3bc7a8 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.910663 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.911328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.911695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.911971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.945774 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.984746 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.986135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.087170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 613/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:46:40.093666 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.113272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.197795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc1ed-b92e-438f-987d-63d0882b29cc None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.231998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.232791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.233071 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.233339 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.243550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.244932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:40.263582 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:40.265014 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:40.267072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:40.267072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:40.267072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:40.267072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:40.267072 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 602/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:46:40.272142 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.337421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.377404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150a24b-b425-45c2-b903-a9f5a241b8c0 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.410246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.410894 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.411174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.411446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.412159 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:40.412460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.413807 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.425196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 614/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:40.431070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.500875 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.532191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875ef1-81f5-48bc-bdc7-c07da41b40ec None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.573993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.574935 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.575717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.575999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.583741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.585393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:40.600635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:40.601715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:40.602651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 603/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:40.608012 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.711621 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a20a93-0b09-4e66-bd1e-e0b08324dbed None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.737099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.748924 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.749581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.749868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.750175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.751000 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:40.751257 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.752666 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d2211-1913-42e2-b4d6-da1dcbcac3fe None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.764618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 615/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:40.770957 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.871206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6b798-6bde-4574-8452-10b2dbf351e4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.900031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.905623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.906290 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.906568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.906836 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.907656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=1e8c0d751eff42e9aa566ff36c19db84) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:40.907959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.909379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723d1c3-01dd-4383-8dc1-1717a3f6fc56 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.914593 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 604/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/role_assignments?user.id=1e8c0d751eff42e9aa566ff36c19db84&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:42.924973 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dc358-3f81-4ed4-9a56-0d4119f92cdf None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:42.925513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dc358-3f81-4ed4-9a56-0d4119f92cdf None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:42.925513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dc358-3f81-4ed4-9a56-0d4119f92cdf None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:42.926912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 616/1220] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:42.931959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:42.932452 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:42.932546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:42.951036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:42.952222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:43.019011 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 605/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:43.041064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dc358-3f81-4ed4-9a56-0d4119f92cdf None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.144437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dc358-3f81-4ed4-9a56-0d4119f92cdf None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.178596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.178596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.178596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.178596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.189189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:43.197036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:43.198192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:43.199250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 617/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:43.204956 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.303186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a8f4-b8e8-4d04-a7fb-4b211f56a3a2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.326783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.337317 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.337981 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.338297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.338575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.339347 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] RBAC: Adding query filter params (name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:43.339651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.341015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.349058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 606/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:43.354763 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.398302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.452804 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da8264-21be-43ea-bea9-e900472d7870 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.485352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.486170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.486454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.486724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.497152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.499182 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:43.516010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:43.517155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:43.518504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:43.518504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:43.518504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:43.518504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:43.518504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 618/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:43.522938 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.623881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7926afa-3e25-4b77-a980-bc48a2f9314a None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.658460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.659191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.659502 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.659787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.660513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:43.660835 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.662218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.674502 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 607/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:43.680643 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.776971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.784211 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2184e-1547-42c3-8338-b17afd7d1f15 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.817202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.817987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.818299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.818573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.825903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.827358 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:43.842615 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:43.843631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:43.844896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 619/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:46:43.849629 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.896606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.948749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447e22e-aacc-472f-9257-1e7420ff2506 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.983555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.984237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.984518 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.984795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.985713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:43.986049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.987492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.007454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 608/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:44.013311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.133597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fb4d6-635a-4397-b1b9-5dc42d5687c8 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.169112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.169922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.170230 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.170505 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.180711 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.182227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:44.200022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:44.201660 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:44.202655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:44.202655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:44.202655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:44.202655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:44.202655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 620/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:44.207708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.307345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4a8a9-e5c1-4d6a-abbc-105a776b73a3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.309868 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:44.341092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.342341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.342623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.342893 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.343613 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:44.343917 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.345294 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.356798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 609/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:44.372018 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.376734 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:44.482330 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60964e2-be9b-4f7a-9b87-e656ee8e17fb None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.516969 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.517758 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.518066 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.518340 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.524829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.526285 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:44.544634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:44.545639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:44.546704 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 621/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:44.552437 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.653023 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9b15a-eed0-4af5-8060-e3f30fa5b6eb None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.689132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.689788 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.690103 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.690383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.691174 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:44.691484 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.692913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d2e26-a2ae-4550-bb80-26759c7e4d2b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.705407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 610/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:44.711681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.780250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:44.814719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102f023-a541-4c7d-9450-2b15b0db3d07 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.848840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e2b6-17da-4e0d-a8e1-a2cf59f88c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.849768 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e2b6-17da-4e0d-a8e1-a2cf59f88c79 None admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.850072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e2b6-17da-4e0d-a8e1-a2cf59f88c79 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.850348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e2b6-17da-4e0d-a8e1-a2cf59f88c79 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/1e8c0d751eff42e9aa566ff36c19db84/roles/c8e0b2c1dfc441ceb25f7706165f329b` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.865437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e2b6-17da-4e0d-a8e1-a2cf59f88c79 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=1e8c0d751eff42e9aa566ff36c19db84, role_id=c8e0b2c1dfc441ceb25f7706165f329b)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.872437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e2b6-17da-4e0d-a8e1-a2cf59f88c79 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.970776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 622/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:46:44 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/1e8c0d751eff42e9aa566ff36c19db84/roles/c8e0b2c1dfc441ceb25f7706165f329b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:46:46.965304 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349628b9-4890-4c8e-8798-56a3e9318a58 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:46.965863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349628b9-4890-4c8e-8798-56a3e9318a58 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:46.965863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349628b9-4890-4c8e-8798-56a3e9318a58 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:46.967244 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 611/1233] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:46.972715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:46.973125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:46.973272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:46.991691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:46.992885 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:47.058408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 623/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:47.084337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349628b9-4890-4c8e-8798-56a3e9318a58 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.109676 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.185776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349628b9-4890-4c8e-8798-56a3e9318a58 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.219163 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.219898 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.220184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.220457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.230511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:47.239195 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:47.240284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:47.241370 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 612/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:47.246504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.292614 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.344247 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ad62a-243d-43eb-974e-a5f9d9f2ec1e None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.377332 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.377982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.378294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.378562 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.379299 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:47.379597 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.380938 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.389082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 624/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:47.395186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.479541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.495183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4e43-961f-435f-95ee-3da92d45ae09 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.527062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.527851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.528206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.528483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.539076 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.540572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:47.558048 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:47.559150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:47.560301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:47.560301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:47.560301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:47.560301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:47.560301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 613/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:47.565057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.666180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8aab4-26ba-409b-ad22-d2455f4af7f9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.675655 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.700312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.700982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.701778 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.702078 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.702812 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:47.703117 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.704494 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.716438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 625/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:47.722708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.825455 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80318cd-00dd-4d07-a3fd-adb6f907b860 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.860810 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.861614 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.861894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.862197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.864966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.870139 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.871623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:47.886576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:47.887708 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:47.888720 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 614/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:47.894449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.994394 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23efffe-fca7-48c2-92e2-3873ff516794 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.029189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.029870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.030189 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.030468 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.031317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:48.031627 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.033057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.050918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 626/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:48.057046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.159598 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e410fc9-ecb1-4d1a-b5fd-9f2c4e68f131 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.193393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.194070 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.194356 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.194628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.256536 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.263208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.265217 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.362974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 615/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:46:48.368868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.397204 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.470798 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1c84b-1eef-42e6-82e0-ae2f1a8db076 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.505521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.506411 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.506693 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.506966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.517685 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.519172 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:48.537305 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:48.540433 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:48.541396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:48.541396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:48.541396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:48.541396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:48.541396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 627/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:48.544499 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.657509 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.663476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3e15-8d16-48fb-bdf4-b3381959499b None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.697730 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.698440 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.698729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.699005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.699732 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:48.700036 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.701451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.712795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 616/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:48.719140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.798947 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.820454 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a9ce-0ed7-4f01-a912-387281d2a85b None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.853967 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.854776 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.855061 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.859174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.863005 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.864450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:48.878194 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:48.879255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:48.880312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 628/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:48.888022 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.987641 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f92e53-f6c3-4dfc-bc74-473d714cddf8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:49.020273 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:49.020919 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:49.021193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:49.021462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:49.022271 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:49.022576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:49.023998 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58d387-4024-4713-bfbd-91312d65bba3 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:49.036309 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 617/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:49.042482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:49.139674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3085ec1-f22b-4a0c-87da-3436e2f30446 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:49.173723 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:49.174382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:49.174662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:49.174931 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:49.175744 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] RBAC: Adding query filter params (role.id=c8e0b2c1dfc441ceb25f7706165f329b, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=1e8c0d751eff42e9aa566ff36c19db84) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:49.176047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:49.177474 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583f8a3-ba13-4549-8fa5-2b5812e78359 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:49.185084 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 629/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:46:49 2022] GET /identity/v3/role_assignments?user.id=1e8c0d751eff42e9aa566ff36c19db84&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=c8e0b2c1dfc441ceb25f7706165f329b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:46:51.249087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598e9d-2e35-4c71-9670-611b88d415d2 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.249623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598e9d-2e35-4c71-9670-611b88d415d2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.249623 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598e9d-2e35-4c71-9670-611b88d415d2 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.251088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 618/1247] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:51.256434 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.256858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.256988 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.263736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.276099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:51.277290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:51.342086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 630/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:51.366087 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598e9d-2e35-4c71-9670-611b88d415d2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.383739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.463881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598e9d-2e35-4c71-9670-611b88d415d2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.498109 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.498928 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.499256 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.499481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.509401 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:51.517550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:51.524094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:51.525146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 619/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:51.526061 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.624306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57bbe-199d-487a-a23c-11575c72d8ba None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.653230 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.657976 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.658700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.658994 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.659284 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.660025 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:51.660326 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:51.661752 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:51.669676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 631/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:51.675519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.753066 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.775170 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b56a4-6750-45ce-840b-02a3ca087ed3 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.806697 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.807476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.813568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.813568 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.818808 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:51.820286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:51.837357 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:51.848795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:51.849700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:51.849700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:51.849700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:51.849700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:51.849700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 620/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:51.849993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.942362 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ed843-f3a3-4834-8b92-de7be92e5f50 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.975867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.976523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.976802 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.977072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.977783 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:51.978104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:51.979506 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:51.992765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 632/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:51.997607 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.100084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce510e6a-5ba9-413b-aeaf-76457b0e2d86 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.126581 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.133982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.134811 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.135091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.135362 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.142610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.144037 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:52.157933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:52.159128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:52.160198 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 621/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:52.165066 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.225313 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.269281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4b463-4e82-4990-83c6-b36a537ee037 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.301978 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.302674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.303043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.303545 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.304619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:52.304933 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.306476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.323953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 633/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:52.329687 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.427481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31635fd5-a074-4d8d-ad48-cbcd99d527ea None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.461976 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.462658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.462940 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.463213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.518718 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.531197 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.532651 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.631704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 622/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:46:52.637698 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.739294 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4c87-380a-40d7-8305-126ea9ac74bf None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.753268 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.778612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.779507 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.779791 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.780178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.790567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.792011 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:52.809010 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:52.810121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:52.811077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:52.811077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:52.811077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:52.811077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:52.811077 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 634/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:52.816200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.903143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.915302 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dda10-a215-4b8a-815c-80f23cff3e18 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.948182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.948851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.949133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.949434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.950211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:52.950511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.952424 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.963819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 623/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:52.970243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.074144 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6025-b7b9-4a9d-af6b-dc899b60918d None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.107942 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.108784 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.109060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.109325 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.116081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.117617 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:53.132070 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:53.133174 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:53.134409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 635/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:53.139282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.239672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8cf9c-fc42-4dcb-98e2-fcb7cbe865b4 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.274359 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.275024 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.275318 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.275579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.276369 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:53.276674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.278135 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b0339-affe-4fed-a038-faf7cc1ff1e1 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.290323 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 624/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:53.296287 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.322214 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:53.399706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5380d2-9e50-4010-806e-0b91f1934733 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.434354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.435015 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.435296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.435575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.436922 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=1e8c0d751eff42e9aa566ff36c19db84) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:53.437243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.438690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766fc2d-dae1-47eb-868d-b7d11ab03d1f None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.444256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 636/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:46:53 2022] GET /identity/v3/role_assignments?user.id=1e8c0d751eff42e9aa566ff36c19db84&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:55.496050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b0feb-9bbd-43d5-a90c-fe03fb263e38 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.496592 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b0feb-9bbd-43d5-a90c-fe03fb263e38 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.496851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b0feb-9bbd-43d5-a90c-fe03fb263e38 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.498354 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 625/1261] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:46:55.503616 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.504245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.504245 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.527427 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:55.528736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:55.605684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 637/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:55.629132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b0feb-9bbd-43d5-a90c-fe03fb263e38 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:55.641293 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:55.736595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b0feb-9bbd-43d5-a90c-fe03fb263e38 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:55.771458 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:55.772260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.772544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.772818 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.783838 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:55.791579 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:55.792704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:55.793798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 626/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:55.798990 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:55.817258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:55.900042 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895436d-1c00-4923-8a78-bb26cf40bcba None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:55.941453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:55.942175 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.942476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.942754 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.943513 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:55.943819 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:55.945228 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:55.955123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 638/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:55.961013 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.044058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.078547 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6712755-12a5-41c2-b3ce-8546a0f98fa7 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.113855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.114703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.114985 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.115263 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.126707 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.128165 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:56.148572 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:56.150925 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:56.153191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:56.153191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:56.153191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.153191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:56.153191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 627/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:56.155774 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.241684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.261887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c773-45c2-46c8-9531-8b536800ac42 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.298371 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.298989 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.299269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.299579 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.300317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:56.300622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.304684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.317883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 639/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:56.324636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.427206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2da0d-5048-43a2-b2a2-5316709ff6b8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.447035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.460625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.461481 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.461757 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.462051 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.470383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.471848 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:56.487602 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:56.488777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:56.489866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 628/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:56.495092 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.599736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6350f-dd59-482d-b71e-fba9f3b3a183 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.633676 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.634387 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.634673 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.634949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] PATH_INFO: `/v3/users` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.635792 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:56.636097 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.637565 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.647607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.656314 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 640/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:56.662129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.779814 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fc1a2-ca28-4a3a-ab43-cf39bb5adbe8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.814300 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.815344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.815724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.816133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.827175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.829158 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:56.846977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:56.848132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:56.849254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:56.849254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:56.849254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.849254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:56.849254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 629/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:56.858861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.956409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da2808-8c85-4d96-b415-7634344f1c49 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.992252 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.992919 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.993201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.993471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.994235 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:56.994540 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.995909 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.008192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 641/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:57.014213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.046865 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:57.119564 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd995-0b45-4d3a-a968-4ab381fe36e8 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.153515 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.154483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.154769 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.155047 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.162921 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.164384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:57.180092 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:57.181189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:46:57.182368 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 630/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:57.187450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.249008 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:57.291531 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f677031-074e-4d12-8d2f-bc604123cc71 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.327090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.331853 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.332140 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.332421 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.333234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:57.333546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.335014 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea42b2-d9e2-4586-a5a2-3607d3b745f5 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.348627 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 642/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:57.355136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.448904 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:57.458445 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018353a-d91a-46fe-bd2b-9e6e08be8e31 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.492975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dcf98-cc9f-4c22-a604-dc466579ccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.493863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dcf98-cc9f-4c22-a604-dc466579ccb9 None admin] REQUEST_METHOD: `PUT` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.494173 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dcf98-cc9f-4c22-a604-dc466579ccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.494454 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dcf98-cc9f-4c22-a604-dc466579ccb9 None admin] PATH_INFO: `/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/1e8c0d751eff42e9aa566ff36c19db84/roles/6d918ee96a11428b9d8a7c436e58eb31` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.496469 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dcf98-cc9f-4c22-a604-dc466579ccb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=47f153d7ef574e89b303a2d0b529fb40, user_id=1e8c0d751eff42e9aa566ff36c19db84, role_id=6d918ee96a11428b9d8a7c436e58eb31)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.498138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dcf98-cc9f-4c22-a604-dc466579ccb9 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.641441 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 631/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 25 18:46:57 2022] PUT /identity/v3/projects/47f153d7ef574e89b303a2d0b529fb40/users/1e8c0d751eff42e9aa566ff36c19db84/roles/6d918ee96a11428b9d8a7c436e58eb31 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:46:59.683224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aacb72-dee4-49f5-a262-0a6f5fd90b60 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:59.683769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aacb72-dee4-49f5-a262-0a6f5fd90b60 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:59.683869 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aacb72-dee4-49f5-a262-0a6f5fd90b60 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:59.685265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 643/1274] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:59.691094 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:59.691533 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:59.691644 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:59.709604 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:59.710987 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:59.778140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 632/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:59.802445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aacb72-dee4-49f5-a262-0a6f5fd90b60 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:59.904684 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aacb72-dee4-49f5-a262-0a6f5fd90b60 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:59.923997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:59.940265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:59.941056 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:59.941338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:59.941613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:59.951462 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:59.959191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:59.960173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:46:59.961072 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 644/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 25 18:46:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:59.966606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.055982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.075399 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb528e-8cb9-4c51-8f4e-e29f13653aec None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.112148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.112827 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.113134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.113391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.114179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:00.114487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.115892 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.124313 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 633/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 25 18:46:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:00.130485 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.235178 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccca3c-9b76-458f-b970-d3bc7993cf5c None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.267653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.268434 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.268717 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.268994 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.279312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.280737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:47:00.298106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:00.299167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:00.300112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:47:00.300112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:47:00.300112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:00.300112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:00.300112 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 645/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:00.305061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.404235 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeab7e5-311d-4d72-9cd1-06f52f8d0daa None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.439768 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.440587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.440785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.441065 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.441792 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:00.442126 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.443554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.455760 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 634/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:00.461448 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.492225 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.563367 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cce5e-1492-4fc4-b64d-2f2bff3bd8ed None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.599770 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.600570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.600854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.601129 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.609543 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.611197 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:47:00.626432 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:00.627399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:00.628404 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 646/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:00.633569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.640212 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.730222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e188c8c-137a-42d3-aa6d-fc1cbdd56104 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.767000 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.767679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.767966 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.768243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.769124 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:00.769436 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.771016 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.789580 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 635/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:00.794746 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.887878 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.898196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74a36-c7c9-4341-b49e-5e7f1daada45 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.941405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.942082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.942373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.942649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.010491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.011950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.115318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.128123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 647/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:47:01.133958 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.194826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.232371 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305e7e4-9253-49dc-8f07-cac95d9ad809 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.266910 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.267636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.267920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.268196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.279122 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.280492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:47:01.298485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:01.315221 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:01.316434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:47:01.316434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:47:01.316434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:01.316434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:47:01.316434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 636/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:01.316805 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.403700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22231f1-c569-4bf2-8bcf-dcca21cdafa9 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.443626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.444298 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.444581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.444857 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] PATH_INFO: `/v3/domains` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.445641 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:01.445947 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.457111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.459255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 648/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:47:01.465031 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.565671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95feda8e-1dd8-4586-ac86-50134063f9a2 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.580516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.600625 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.606295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.606295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.606295 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.608435 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.610799 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:47:01.624246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:01.625320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: ERROR keystone.server.flask.application Jul 25 18:47:01.626353 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 637/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:01.631466 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.681691 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.738500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5990b59-b159-4382-9188-5bbf5cacc6dd None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.775198 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.775866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.776239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.776518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.777302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:01.777608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.779049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315d32-1482-42a0-bb90-19c05689c4a1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.795639 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 649/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:01.801852 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.912738 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf6ec-246c-452d-bcb3-0d748f3c9637 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.947924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.948578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.948862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.949141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.949971 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] RBAC: Adding query filter params (role.id=6d918ee96a11428b9d8a7c436e58eb31, scope.project.id=47f153d7ef574e89b303a2d0b529fb40, user.id=1e8c0d751eff42e9aa566ff36c19db84) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:01.950314 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.951754 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94f3d2-edbf-492e-9271-b5d120e75999 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.958106 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 638/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/role_assignments?user.id=1e8c0d751eff42e9aa566ff36c19db84&scope.project.id=47f153d7ef574e89b303a2d0b529fb40&role.id=6d918ee96a11428b9d8a7c436e58eb31 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:04.023378 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8851-7679-4ec6-a65f-d3f8e5e25a08 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.023943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8851-7679-4ec6-a65f-d3f8e5e25a08 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.023943 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8851-7679-4ec6-a65f-d3f8e5e25a08 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.025303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 650/1288] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:04.030901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.031315 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.031459 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.051017 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:04.053713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:04.082565 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:04.120555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 639/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:04.142951 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c8851-7679-4ec6-a65f-d3f8e5e25a08 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:04.249020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c8851-7679-4ec6-a65f-d3f8e5e25a08 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:04.260349 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:04.290186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:04.290186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.290186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.290186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.290649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:04.290649 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:04.301716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:04.303037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:04.304118 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 651/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:47:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:04.309299 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:04.412744 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b16db-3967-4669-9f7c-4948cfb8b0a5 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:04.449456 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:04.450164 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.450450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.450728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.451476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:04.451783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:04.453182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724de60b-c27d-400f-9cfb-55a1ceaf8566 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:04.458960 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 640/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:04.464843 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:04.569737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3b0ce-9412-4a63-8982-8c3b2af36b09 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:04.606785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:04.607489 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.607764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.608035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] PATH_INFO: `/v3/services` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.608759 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:04.609168 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:04.611487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f71acd-5758-448b-b8cf-1b7fb5596a86 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:04.616949 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 652/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:06.616253 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450f7ce-3ffe-4a39-8e74-ee2f5d3788d8 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:06.616770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450f7ce-3ffe-4a39-8e74-ee2f5d3788d8 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:06.616879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450f7ce-3ffe-4a39-8e74-ee2f5d3788d8 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:06.618194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 641/1293] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:06.623601 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807bb66-f535-4df7-85db-9ce7687ab9eb None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:06.624045 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807bb66-f535-4df7-85db-9ce7687ab9eb None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:06.624173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807bb66-f535-4df7-85db-9ce7687ab9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:06.642361 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b807bb66-f535-4df7-85db-9ce7687ab9eb None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:06.643608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b807bb66-f535-4df7-85db-9ce7687ab9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:06.675913 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:06.714519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 653/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:06.738241 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450f7ce-3ffe-4a39-8e74-ee2f5d3788d8 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:06.769879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:06.842450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450f7ce-3ffe-4a39-8e74-ee2f5d3788d8 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:06.877598 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536f8f5-197b-4965-9870-07094937f867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:06.878297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536f8f5-197b-4965-9870-07094937f867 None admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:06.878663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536f8f5-197b-4965-9870-07094937f867 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:06.878939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536f8f5-197b-4965-9870-07094937f867 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:06.879656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536f8f5-197b-4965-9870-07094937f867 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:06.881051 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536f8f5-197b-4965-9870-07094937f867 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:06.990909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 642/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 25 18:47:06 2022] POST /identity/v3/services => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:47:09.068714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805078e7-7560-417e-acb5-4d7ec52f4f3c None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.069569 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805078e7-7560-417e-acb5-4d7ec52f4f3c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.069736 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805078e7-7560-417e-acb5-4d7ec52f4f3c None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.071264 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 654/1296] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:09.077464 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.078027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.078138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.098293 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:09.100773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:09.179700 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 643/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:09.202105 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805078e7-7560-417e-acb5-4d7ec52f4f3c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.313021 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805078e7-7560-417e-acb5-4d7ec52f4f3c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:09.349307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:09.350099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.350379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.355535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] PATH_INFO: `/v3/services/placement` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.355535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:09.355535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:09.365267 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:09.369436 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:09.370504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 655/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:09.372676 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.392437 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:09.481681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c743ae9-820f-4443-b46c-83e93c15a235 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:09.518116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:09.518783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.519111 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.519384 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.520102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:09.520406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:09.521791 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:09.527293 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 644/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:09.533303 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.566192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:09.637456 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dedaa-be13-445c-bdf0-48114db0e68a None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:09.672544 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:09.673784 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.674096 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.674381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.675170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] RBAC: Adding query filter params (interface=public, service_id=88eb243e74674661acf64245f2bf057c, region_id=RegionOne) {{(pid=86941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:09.675479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:09.676834 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13aee1-9ae9-41b3-84e0-ccb4dd41ff83 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:09.682035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 656/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/endpoints?service_id=88eb243e74674661acf64245f2bf057c&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:09.687828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.770733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:09.788980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119176e-4eb4-42f8-a8a7-e7f3e9740f79 None admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:09.822449 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b76eb-3f9d-47e8-b300-12079a1d2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:09.823126 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b76eb-3f9d-47e8-b300-12079a1d2807 None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.823413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b76eb-3f9d-47e8-b300-12079a1d2807 None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.823683 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b76eb-3f9d-47e8-b300-12079a1d2807 None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.824406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b76eb-3f9d-47e8-b300-12079a1d2807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:09.825778 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b76eb-3f9d-47e8-b300-12079a1d2807 None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:09.837148 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 645/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:47:11.863671 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2bfd2-1ead-47bf-a102-0b41081af4ad None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:11.864188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2bfd2-1ead-47bf-a102-0b41081af4ad None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:11.864188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2bfd2-1ead-47bf-a102-0b41081af4ad None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:11.865521 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 657/1302] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:11.870846 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:11.871259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:11.871434 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:11.890142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:11.891346 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:11.962544 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 646/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:11.988148 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2bfd2-1ead-47bf-a102-0b41081af4ad None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:12.091370 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:12.098936 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2bfd2-1ead-47bf-a102-0b41081af4ad None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:12.135912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:12.136634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.136923 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.137195 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.137851 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:12.139253 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:12.151529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:12.153563 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: ERROR keystone.server.flask.application Jul 25 18:47:12.155233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 658/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 25 18:47:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:12.163682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:12.257612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:12.280254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a7059-1d5f-4199-96e6-cf7b95a937c7 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:12.329091 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:12.329834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.330150 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.330420 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] PATH_INFO: `/v3/services` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.331138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] RBAC: Adding query filter params (name=placement) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:12.331447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:12.332826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4379b2b-60fd-4883-a9c1-d88f221e150c None admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:12.339120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 647/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:12.347796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:12.461564 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6230-1d2f-4d87-8cbf-fcf2e9dc1ff1 None admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:12.501272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6eb7b-180a-4a32-b76b-0592cab1aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:12.502504 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6eb7b-180a-4a32-b76b-0592cab1aec4 None admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.502788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6eb7b-180a-4a32-b76b-0592cab1aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.503064 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6eb7b-180a-4a32-b76b-0592cab1aec4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.503874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6eb7b-180a-4a32-b76b-0592cab1aec4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:12.505293 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6eb7b-180a-4a32-b76b-0592cab1aec4 None admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:12.625719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 659/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:47:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:47:22.477459 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=30 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:22.620694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:20.654717 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701da05-fc32-49bd-a0f7-0fa4169448a0 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:20.655825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701da05-fc32-49bd-a0f7-0fa4169448a0 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:20.656410 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701da05-fc32-49bd-a0f7-0fa4169448a0 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:20.666687 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1742c-2558-47cd-85cc-6680d9ae1084 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:20.668270 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1742c-2558-47cd-85cc-6680d9ae1084 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:20.668813 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1742c-2558-47cd-85cc-6680d9ae1084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:20.671489 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 648/1307] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:20 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:20.783897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f5f1742c-2558-47cd-85cc-6680d9ae1084 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:20.785133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f5f1742c-2558-47cd-85cc-6680d9ae1084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:21.045669 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 660/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:54:21.097741 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866533-0656-48e7-83da-305f846f27be None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:21.098042 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866533-0656-48e7-83da-305f846f27be None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:21.098635 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866533-0656-48e7-83da-305f846f27be None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:21.100908 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 649/1309] 10.222.0.57 () {54 vars in 965 bytes} [Mon Jul 25 18:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:21.108587 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dccb9b-526e-45fd-a6b7-bd7f4894afc0 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:21.108845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dccb9b-526e-45fd-a6b7-bd7f4894afc0 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:21.109083 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dccb9b-526e-45fd-a6b7-bd7f4894afc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:21.190302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-85dccb9b-526e-45fd-a6b7-bd7f4894afc0 None None] MFA Rules not processed for user `4c21d403c2d5498e99eaf13ae72d55ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:21.191547 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-85dccb9b-526e-45fd-a6b7-bd7f4894afc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:21.213712 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:21.273571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 661/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Jul 25 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:21.283202 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866533-0656-48e7-83da-305f846f27be None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:21.285109 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-60866533-0656-48e7-83da-305f846f27be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:21.410576 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866533-0656-48e7-83da-305f846f27be None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:21.497802 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:21.498504 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:21.498787 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:21.499061 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:21.500761 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:21.568421 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:21.628783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:21.630282 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d580-8030-4819-b1c3-3fa169900048 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:21.649108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 650/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:54:23.975490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e047b9-60cc-4093-b3c7-8384b5d518f2 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:23.976049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e047b9-60cc-4093-b3c7-8384b5d518f2 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:23.976049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e047b9-60cc-4093-b3c7-8384b5d518f2 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:23.977634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 662/1312] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:23.983059 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab916-6125-49a1-8cc4-5fabc91ee463 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:23.983596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab916-6125-49a1-8cc4-5fabc91ee463 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:23.983596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab916-6125-49a1-8cc4-5fabc91ee463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:24.069924 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-361ab916-6125-49a1-8cc4-5fabc91ee463 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:24.071596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-361ab916-6125-49a1-8cc4-5fabc91ee463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:24.190552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 651/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:24.245486 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb227581-50a9-492e-9031-8ea930685d62 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:24.245797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb227581-50a9-492e-9031-8ea930685d62 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:24.246109 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb227581-50a9-492e-9031-8ea930685d62 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:24.247720 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 663/1314] 10.222.0.57 () {54 vars in 965 bytes} [Mon Jul 25 18:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:24.254968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07439-f091-4c97-b4aa-68d954571257 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:24.255285 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07439-f091-4c97-b4aa-68d954571257 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:24.255545 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07439-f091-4c97-b4aa-68d954571257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:24.277335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:24.358139 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-3ff07439-f091-4c97-b4aa-68d954571257 None None] MFA Rules not processed for user `4c21d403c2d5498e99eaf13ae72d55ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:24.359495 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-3ff07439-f091-4c97-b4aa-68d954571257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:24.453079 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 652/1315] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Jul 25 18:54:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:24.462090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb227581-50a9-492e-9031-8ea930685d62 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:24.463296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-bb227581-50a9-492e-9031-8ea930685d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:24.581681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb227581-50a9-492e-9031-8ea930685d62 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:24.664397 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:24.669460 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:24.670142 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:24.670420 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:24.670695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:24.672383 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:24.813845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:24.815317 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fdf2b-948f-4d1a-ad62-0c8ef91fd13e service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:24.835548 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 664/1316] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:27.724909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ec585-6385-417b-ac31-60ab92aa82eb None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:27.725516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ec585-6385-417b-ac31-60ab92aa82eb None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:27.725516 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ec585-6385-417b-ac31-60ab92aa82eb None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:27.726884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 653/1317] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:27.732476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a407-60ff-409a-9764-5b4adcc9bfad None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:27.732928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a407-60ff-409a-9764-5b4adcc9bfad None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:27.733041 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a407-60ff-409a-9764-5b4adcc9bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:27.797618 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-6fc2a407-60ff-409a-9764-5b4adcc9bfad None None] MFA Rules not processed for user `9134c5ce0660415e9a3bc4b4ecb08e82`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:27.798758 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-6fc2a407-60ff-409a-9764-5b4adcc9bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:27.850497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:27.895244 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 665/1318] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:27.935501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ec585-6385-417b-ac31-60ab92aa82eb None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:27.937046 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1a1ec585-6385-417b-ac31-60ab92aa82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:28.050205 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ec585-6385-417b-ac31-60ab92aa82eb None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:28.052968 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:28.125907 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:28.126594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:28.126879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:28.127155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:28.128815 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:28.240546 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:28.242004 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc7b1b-2314-4b27-bae6-53f292004f8d service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:28.261099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 654/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:32.714478 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f2d6e-a2bf-48a8-a590-d60c982986b2 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:32.714997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f2d6e-a2bf-48a8-a590-d60c982986b2 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:32.714997 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f2d6e-a2bf-48a8-a590-d60c982986b2 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:32.716311 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 666/1320] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:32.721702 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d838fa-81a7-45e6-bb09-7104a80bea91 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:32.722192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d838fa-81a7-45e6-bb09-7104a80bea91 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:32.722292 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d838fa-81a7-45e6-bb09-7104a80bea91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:32.764778 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:32.791713 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-41d838fa-81a7-45e6-bb09-7104a80bea91 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:32.792991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-41d838fa-81a7-45e6-bb09-7104a80bea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:32.892443 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 655/1321] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:32.932573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f2d6e-a2bf-48a8-a590-d60c982986b2 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:32.934214 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0d9f2d6e-a2bf-48a8-a590-d60c982986b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:33.041883 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f2d6e-a2bf-48a8-a590-d60c982986b2 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:33.120446 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:33.121107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:33.121390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:33.121665 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:33.123392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:33.131900 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:33.247155 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:33.248695 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17062cb7-193d-499d-87ce-522d0f3810f7 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:33.268761 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 667/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:36.952663 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf12ad-361c-41c3-820a-965e075e6d95 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:36.953206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf12ad-361c-41c3-820a-965e075e6d95 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:36.953206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf12ad-361c-41c3-820a-965e075e6d95 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:36.954725 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 656/1323] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:36.960201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208ddfb-fc19-4932-8d7c-e49b1cda4cdc None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:36.960653 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208ddfb-fc19-4932-8d7c-e49b1cda4cdc None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:36.960745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208ddfb-fc19-4932-8d7c-e49b1cda4cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:37.033229 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-a208ddfb-fc19-4932-8d7c-e49b1cda4cdc None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:37.034733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a208ddfb-fc19-4932-8d7c-e49b1cda4cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:37.117607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:37.167908 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 668/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:37.206578 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf12ad-361c-41c3-820a-965e075e6d95 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:37.208068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1cdf12ad-361c-41c3-820a-965e075e6d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:37.324009 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf12ad-361c-41c3-820a-965e075e6d95 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:37.336864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:37.429176 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:37.429837 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:37.430153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:37.430425 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:37.432174 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:37.620050 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:37.622963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fe0bb-0297-4c23-af3b-84ffd9bd53c5 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:37.662057 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 657/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:39.989467 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfac571-6bb8-45f5-9dae-2515ef3ca841 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:39.990101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfac571-6bb8-45f5-9dae-2515ef3ca841 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:39.990471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfac571-6bb8-45f5-9dae-2515ef3ca841 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:39.993130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 669/1326] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:40.003209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e7da-f85a-42e1-98fc-ea93aa5ca0fa None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:40.003850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e7da-f85a-42e1-98fc-ea93aa5ca0fa None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:40.004188 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e7da-f85a-42e1-98fc-ea93aa5ca0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:40.076123 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:40.101619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-e850e7da-f85a-42e1-98fc-ea93aa5ca0fa None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:40.103012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-e850e7da-f85a-42e1-98fc-ea93aa5ca0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:40.243117 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 658/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:40.289881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfac571-6bb8-45f5-9dae-2515ef3ca841 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:40.292050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-3cfac571-6bb8-45f5-9dae-2515ef3ca841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:40.421584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfac571-6bb8-45f5-9dae-2515ef3ca841 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:40.535874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:40.537013 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:40.537519 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:40.538035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:40.541642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:40.567525 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:40.680250 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:40.681738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a3e2c-6d8e-4ddc-9820-9bddad1e1d06 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:40.703622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 670/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:54:42.849035 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb816d0-e23b-4b20-b305-926ffe6a331e None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:42.849649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb816d0-e23b-4b20-b305-926ffe6a331e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:42.849649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb816d0-e23b-4b20-b305-926ffe6a331e None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:42.850954 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 659/1329] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:42.857483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac386a-8f17-4fce-997f-a93c62278952 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:42.858324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac386a-8f17-4fce-997f-a93c62278952 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:42.858324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac386a-8f17-4fce-997f-a93c62278952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:42.938354 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-cbac386a-8f17-4fce-997f-a93c62278952 None None] MFA Rules not processed for user `9134c5ce0660415e9a3bc4b4ecb08e82`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:42.939690 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-cbac386a-8f17-4fce-997f-a93c62278952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:43.011590 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:43.077731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 671/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:54:43.116448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb816d0-e23b-4b20-b305-926ffe6a331e None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:43.118129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8fb816d0-e23b-4b20-b305-926ffe6a331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:43.256885 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb816d0-e23b-4b20-b305-926ffe6a331e None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:43.282881 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:43.344185 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:43.344850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:43.345151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:43.345407 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:43.347183 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:43.479945 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:43.481332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c407c2-9b6a-4820-9099-eff286ca70c9 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:43.502873 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 660/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:46.193172 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7815d32-9440-4536-8e1b-9727c82a0a46 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:46.193740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7815d32-9440-4536-8e1b-9727c82a0a46 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:46.193740 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7815d32-9440-4536-8e1b-9727c82a0a46 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:46.195129 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 672/1332] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:46.200795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29791-1c29-42b1-88bc-8e2150dedcc7 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:46.201209 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29791-1c29-42b1-88bc-8e2150dedcc7 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:46.201349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29791-1c29-42b1-88bc-8e2150dedcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:46.266508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:46.269380 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-d9a29791-1c29-42b1-88bc-8e2150dedcc7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:46.270662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-d9a29791-1c29-42b1-88bc-8e2150dedcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:46.399548 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 661/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:46.436090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7815d32-9440-4536-8e1b-9727c82a0a46 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:46.437646 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a7815d32-9440-4536-8e1b-9727c82a0a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:46.545600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7815d32-9440-4536-8e1b-9727c82a0a46 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:46.620033 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:46.620680 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:46.620958 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:46.621230 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:46.622920 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:46.650737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:46.733173 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:46.734647 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbcf23-cdc6-4975-981b-1b5dad9be9dd service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:46.753551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 673/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:48.859038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b8245-ebf1-4efb-9de6-1ec4423032cf None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:48.859879 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b8245-ebf1-4efb-9de6-1ec4423032cf None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:48.860063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b8245-ebf1-4efb-9de6-1ec4423032cf None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:48.861554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 662/1335] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:48.866890 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c7d2-9d69-432e-be0e-e4f58361c590 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:48.867351 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c7d2-9d69-432e-be0e-e4f58361c590 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:48.867452 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c7d2-9d69-432e-be0e-e4f58361c590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:48.938207 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-df84c7d2-9d69-432e-be0e-e4f58361c590 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:48.939425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-df84c7d2-9d69-432e-be0e-e4f58361c590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:49.018941 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:49.052786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 674/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:49.094152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b8245-ebf1-4efb-9de6-1ec4423032cf None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:49.095699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-0f6b8245-ebf1-4efb-9de6-1ec4423032cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:49.202786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b8245-ebf1-4efb-9de6-1ec4423032cf None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:49.232900 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:49.279193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:49.279850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:49.280132 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:49.280404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:49.282099 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:49.391667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:49.393110 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28ecb-8758-4046-a70a-ad8d716f7f9c service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:49.411774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 663/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:52.131163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c9cce-0455-4bd2-8858-4fdb4dbeccd1 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:52.131731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c9cce-0455-4bd2-8858-4fdb4dbeccd1 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:52.131731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c9cce-0455-4bd2-8858-4fdb4dbeccd1 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:52.133090 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 675/1338] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:52.138557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c476ec-30e4-4c4b-b18f-53e9db7d1673 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:52.139033 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c476ec-30e4-4c4b-b18f-53e9db7d1673 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:52.139130 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c476ec-30e4-4c4b-b18f-53e9db7d1673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:52.208770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-90c476ec-30e4-4c4b-b18f-53e9db7d1673 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:52.209990 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-90c476ec-30e4-4c4b-b18f-53e9db7d1673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:52.219088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:52.308437 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 664/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:52.348900 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c9cce-0455-4bd2-8858-4fdb4dbeccd1 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:52.350524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f65c9cce-0455-4bd2-8858-4fdb4dbeccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:52.461870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c9cce-0455-4bd2-8858-4fdb4dbeccd1 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:52.539139 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:52.540226 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:52.540508 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:52.540821 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:52.542604 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:52.583324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:52.671751 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:52.673200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822f1ea-b1d9-4a88-9021-7704027bf796 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:52.692964 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 676/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:54:57.147646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7433-fa66-4534-8552-021ecffd9c08 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:57.148181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7433-fa66-4534-8552-021ecffd9c08 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:57.148181 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7433-fa66-4534-8552-021ecffd9c08 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:57.149517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 665/1341] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:57.156205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e2e35-24dd-4094-af97-09712a543f7e None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:57.156756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e2e35-24dd-4094-af97-09712a543f7e None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:57.156756 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e2e35-24dd-4094-af97-09712a543f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:57.241352 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-e61e2e35-24dd-4094-af97-09712a543f7e None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:57.242612 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-e61e2e35-24dd-4094-af97-09712a543f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:57.349920 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:57.366060 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 677/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:57.406320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7433-fa66-4534-8552-021ecffd9c08 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:57.407887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-93ae7433-fa66-4534-8552-021ecffd9c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:57.529569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7433-fa66-4534-8552-021ecffd9c08 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:57.575746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:57.613699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:57.614615 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:57.614989 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:57.615511 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:57.618243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:57.734961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:57.736412 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af4fd-a20f-4473-be84-66b8ab4335b7 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:57.756096 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 666/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:00.852381 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8720f-a46e-44da-8354-b52c864cf77d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:00.853021 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8720f-a46e-44da-8354-b52c864cf77d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:00.853021 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8720f-a46e-44da-8354-b52c864cf77d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:00.854607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 678/1344] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:00.860101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b746a3-afba-46c5-9a52-f260e36984ab None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:00.860834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b746a3-afba-46c5-9a52-f260e36984ab None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:00.860834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b746a3-afba-46c5-9a52-f260e36984ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:00.936681 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-19b746a3-afba-46c5-9a52-f260e36984ab None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:00.937939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-19b746a3-afba-46c5-9a52-f260e36984ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:00.960448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:01.045929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 667/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:01.088280 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8720f-a46e-44da-8354-b52c864cf77d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:01.089828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-01b8720f-a46e-44da-8354-b52c864cf77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:01.200503 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8720f-a46e-44da-8354-b52c864cf77d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:01.282879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:01.283545 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:01.283821 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:01.284086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:01.285820 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:01.332273 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:01.405912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:01.407419 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be628a-c4ce-4de5-8aff-a982ab9f2aa9 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:01.428515 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 679/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:05.820444 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850847eb-7baf-49a2-94a4-31314fabbc77 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:05.820943 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850847eb-7baf-49a2-94a4-31314fabbc77 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:05.821056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850847eb-7baf-49a2-94a4-31314fabbc77 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:05.822755 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 668/1347] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:05.828031 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa71aa7-24b1-4fb2-af2a-771edfc8cb65 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:05.828470 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa71aa7-24b1-4fb2-af2a-771edfc8cb65 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:05.828573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa71aa7-24b1-4fb2-af2a-771edfc8cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:05.914394 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-4fa71aa7-24b1-4fb2-af2a-771edfc8cb65 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:05.916201 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-4fa71aa7-24b1-4fb2-af2a-771edfc8cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:06.029188 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:06.031915 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 680/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:06.071381 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850847eb-7baf-49a2-94a4-31314fabbc77 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:06.073786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-850847eb-7baf-49a2-94a4-31314fabbc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:06.194505 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850847eb-7baf-49a2-94a4-31314fabbc77 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:06.246143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:06.274139 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:06.274833 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:06.275643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:06.275915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:06.277933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:06.405138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:06.406650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab540a-7035-4a12-b518-c5eb53cd2e56 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:06.426863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 669/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:11.650584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd7e1d-ae7d-40f9-8ddc-bfa9725c7724 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:11.651106 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd7e1d-ae7d-40f9-8ddc-bfa9725c7724 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:11.651215 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd7e1d-ae7d-40f9-8ddc-bfa9725c7724 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:11.652983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 681/1350] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 25 18:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:11.659795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774b14-6841-4f3f-8dbd-9f17bfa82c27 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:11.659795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774b14-6841-4f3f-8dbd-9f17bfa82c27 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:11.667701 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774b14-6841-4f3f-8dbd-9f17bfa82c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:11.813062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-4a774b14-6841-4f3f-8dbd-9f17bfa82c27 None None] MFA Rules not processed for user `1e8c0d751eff42e9aa566ff36c19db84`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:11.814349 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-4a774b14-6841-4f3f-8dbd-9f17bfa82c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:11.878776 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:12.071473 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 670/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 25 18:55:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:16.575065 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ea205-2bb4-4604-8d6e-06b02a5f73c0 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:16.575667 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ea205-2bb4-4604-8d6e-06b02a5f73c0 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:16.575667 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ea205-2bb4-4604-8d6e-06b02a5f73c0 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:16.577328 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 682/1352] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 25 18:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:16.583058 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a028-6d10-447b-a85d-6811c0afdc5e None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:16.583804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a028-6d10-447b-a85d-6811c0afdc5e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:16.583804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a028-6d10-447b-a85d-6811c0afdc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:16.736000 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-5f52a028-6d10-447b-a85d-6811c0afdc5e None None] MFA Rules not processed for user `1e8c0d751eff42e9aa566ff36c19db84`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:16.737230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-5f52a028-6d10-447b-a85d-6811c0afdc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:16.821055 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:16.859179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 671/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 25 18:55:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:17.958357 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5697e-257e-4076-a05b-5c83b5f115e2 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:17.958870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5697e-257e-4076-a05b-5c83b5f115e2 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:17.958870 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5697e-257e-4076-a05b-5c83b5f115e2 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:17.960239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 683/1354] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 25 18:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:17.965969 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795a428-542a-499d-b256-e9a739f32a34 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:17.966522 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795a428-542a-499d-b256-e9a739f32a34 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:17.966668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795a428-542a-499d-b256-e9a739f32a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:18.051316 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-0795a428-542a-499d-b256-e9a739f32a34 None None] MFA Rules not processed for user `1e8c0d751eff42e9aa566ff36c19db84`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:18.052541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-0795a428-542a-499d-b256-e9a739f32a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:18.172056 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 672/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 25 18:55:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:22.013986 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6b4d-f11f-4bbc-b3c4-bb8ea92f6839 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.014558 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6b4d-f11f-4bbc-b3c4-bb8ea92f6839 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.014833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6b4d-f11f-4bbc-b3c4-bb8ea92f6839 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.016239 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 684/1356] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:22.021783 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4944a-0204-441e-b7ef-8d2c9de09210 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.022222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4944a-0204-441e-b7ef-8d2c9de09210 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.022514 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4944a-0204-441e-b7ef-8d2c9de09210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.073897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:22.114131 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-9ba4944a-0204-441e-b7ef-8d2c9de09210 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:22.115820 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-9ba4944a-0204-441e-b7ef-8d2c9de09210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.241953 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 673/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:22.297889 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e59d4-ea96-4fe9-b45c-12cded6fa6c0 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.298483 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e59d4-ea96-4fe9-b45c-12cded6fa6c0 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.298968 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e59d4-ea96-4fe9-b45c-12cded6fa6c0 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.301862 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 685/1358] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 25 18:55:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:22.309490 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b19be-6630-4fc5-a0bb-96a451d0ed1a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.310069 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b19be-6630-4fc5-a0bb-96a451d0ed1a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.317699 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b19be-6630-4fc5-a0bb-96a451d0ed1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.414662 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-f94b19be-6630-4fc5-a0bb-96a451d0ed1a None None] MFA Rules not processed for user `2135d214a8274e60ac795f71a0697433`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:22.416733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-f94b19be-6630-4fc5-a0bb-96a451d0ed1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.474426 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:22.559911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 674/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 25 18:55:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:22.573297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e59d4-ea96-4fe9-b45c-12cded6fa6c0 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:22.576061 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-347e59d4-ea96-4fe9-b45c-12cded6fa6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.765390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e59d4-ea96-4fe9-b45c-12cded6fa6c0 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:22.843349 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:22.886465 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:22.887837 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.888170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.888500 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.890237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.997719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:22.999428 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33295a-f493-461b-9a94-fa2c1d48aa9f service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:23.027372 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 686/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:25.326912 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f157-35b1-420e-8f7f-2ef71781995d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:25.327519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f157-35b1-420e-8f7f-2ef71781995d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:25.327519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f157-35b1-420e-8f7f-2ef71781995d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:25.329284 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 675/1361] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:25.335123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dceb5-4a5a-4d31-8612-52f572376545 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:25.335642 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dceb5-4a5a-4d31-8612-52f572376545 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:25.335829 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dceb5-4a5a-4d31-8612-52f572376545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:25.405700 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f42dceb5-4a5a-4d31-8612-52f572376545 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:25.407091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f42dceb5-4a5a-4d31-8612-52f572376545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:25.446208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:25.514062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 687/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:25.560634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2f157-35b1-420e-8f7f-2ef71781995d None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:25.562264 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a2b2f157-35b1-420e-8f7f-2ef71781995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:25.751379 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2f157-35b1-420e-8f7f-2ef71781995d None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:25.762963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:25.924649 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:25.925297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:25.925577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:25.925848 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:25.927557 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:26.033116 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:26.034806 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f29177-ea79-4b19-98a0-28cad015dd77 service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:26.056693 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 676/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:28.609529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cd8da-81bb-49c5-9907-6eb52fa5375e None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.610094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cd8da-81bb-49c5-9907-6eb52fa5375e None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.610094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cd8da-81bb-49c5-9907-6eb52fa5375e None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.611575 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 688/1364] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:28.617088 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f351-eac6-4eeb-9334-2293309f1df5 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.617984 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f351-eac6-4eeb-9334-2293309f1df5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.618301 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f351-eac6-4eeb-9334-2293309f1df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.635634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:28.697429 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-74e5f351-eac6-4eeb-9334-2293309f1df5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:28.698704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-74e5f351-eac6-4eeb-9334-2293309f1df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:28.813366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 677/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:28.861037 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e60e11-4000-4696-be1e-028e78b52c45 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.861402 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e60e11-4000-4696-be1e-028e78b52c45 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.861987 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e60e11-4000-4696-be1e-028e78b52c45 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.863410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 689/1366] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 25 18:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:28.869804 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf0ad5-f90a-4d34-939f-8b80d79b98b2 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.871823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf0ad5-f90a-4d34-939f-8b80d79b98b2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.871823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf0ad5-f90a-4d34-939f-8b80d79b98b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.954487 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-7ccf0ad5-f90a-4d34-939f-8b80d79b98b2 None None] MFA Rules not processed for user `2135d214a8274e60ac795f71a0697433`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:28.955711 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-7ccf0ad5-f90a-4d34-939f-8b80d79b98b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:28.995847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:29.097894 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 678/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 25 18:55:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:29.107422 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e60e11-4000-4696-be1e-028e78b52c45 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:29.109062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-49e60e11-4000-4696-be1e-028e78b52c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:29.280996 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e60e11-4000-4696-be1e-028e78b52c45 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:29.303840 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:29.414137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:29.415124 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:29.415453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:29.415733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:29.417479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:29.520334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:29.521956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7d436-19b6-41d6-8e0c-c99de4e2e242 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:29.549523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 690/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:30.437533 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d9ef-6b57-4613-a222-13b91e9d9de9 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:30.438143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d9ef-6b57-4613-a222-13b91e9d9de9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:30.438143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d9ef-6b57-4613-a222-13b91e9d9de9 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:30.439585 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 679/1369] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:30.444887 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118f4c-49c1-4879-96d0-3aba4c011bc7 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:30.445348 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118f4c-49c1-4879-96d0-3aba4c011bc7 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:30.445442 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118f4c-49c1-4879-96d0-3aba4c011bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:30.479594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:30.519626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-5b118f4c-49c1-4879-96d0-3aba4c011bc7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:30.521147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-5b118f4c-49c1-4879-96d0-3aba4c011bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:30.639573 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 691/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:30.980948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06dc6f-21ad-481c-85a3-29179d9f71ca None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:30.981237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06dc6f-21ad-481c-85a3-29179d9f71ca None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:30.981517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06dc6f-21ad-481c-85a3-29179d9f71ca None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:30.983112 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 680/1371] 10.222.0.57 () {54 vars in 964 bytes} [Mon Jul 25 18:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:30.988028 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddce80-35d5-48d1-9918-b8388b7c36f3 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:30.988318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddce80-35d5-48d1-9918-b8388b7c36f3 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:30.988600 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddce80-35d5-48d1-9918-b8388b7c36f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:31.076782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-18ddce80-35d5-48d1-9918-b8388b7c36f3 None None] MFA Rules not processed for user `aa8107f1366e487f9e2ffc84b1ffb138`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:31.078296 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-18ddce80-35d5-48d1-9918-b8388b7c36f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:31.129232 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:31.159374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 692/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Jul 25 18:55:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:31.170352 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06dc6f-21ad-481c-85a3-29179d9f71ca None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:31.170352 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-5c06dc6f-21ad-481c-85a3-29179d9f71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:31.282451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06dc6f-21ad-481c-85a3-29179d9f71ca None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:31.327661 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:31.367185 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa8107f1366e487f9e2ffc84b1ffb138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:31.367937 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:31.368222 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:31.369034 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:31.371186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:31.496345 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:31.497825 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e764d1-a34c-4c3c-90ec-40f2290a32cf service cinder] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:31.518800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 681/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:31.872633 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148ed6-7c0b-4aed-99d2-3edebad9c538 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:31.873260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148ed6-7c0b-4aed-99d2-3edebad9c538 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:31.873260 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148ed6-7c0b-4aed-99d2-3edebad9c538 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:31.874854 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 693/1374] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:31.883422 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c93a8-c6db-4556-9a47-32a0101a88af None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:31.883723 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c93a8-c6db-4556-9a47-32a0101a88af None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:31.883982 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c93a8-c6db-4556-9a47-32a0101a88af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:31.953658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-881c93a8-c6db-4556-9a47-32a0101a88af None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:31.955051 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-881c93a8-c6db-4556-9a47-32a0101a88af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:31.975961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:32.053789 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 682/1375] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:32.097715 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f148ed6-7c0b-4aed-99d2-3edebad9c538 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:32.099246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-7f148ed6-7c0b-4aed-99d2-3edebad9c538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:32.246868 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f148ed6-7c0b-4aed-99d2-3edebad9c538 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:32.298698 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:32.365251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:32.365959 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:32.366331 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:32.366555 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:32.369059 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:32.464472 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:32.466115 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e050a3-e4dc-4254-bc1a-e4477b8a6150 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:32.485476 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 694/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:34.728785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2895e6-b3fb-43f5-bbde-bdacc175e304 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:34.729320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2895e6-b3fb-43f5-bbde-bdacc175e304 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:34.729320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2895e6-b3fb-43f5-bbde-bdacc175e304 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:34.731038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 683/1377] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:34.736353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daf2d5-0b1e-4e36-876b-5a5ef8b73845 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:34.736748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daf2d5-0b1e-4e36-876b-5a5ef8b73845 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:34.738662 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daf2d5-0b1e-4e36-876b-5a5ef8b73845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:34.828619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-45daf2d5-0b1e-4e36-876b-5a5ef8b73845 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:34.829817 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-45daf2d5-0b1e-4e36-876b-5a5ef8b73845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:34.837365 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:34.929343 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 695/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:34.979637 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2895e6-b3fb-43f5-bbde-bdacc175e304 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:34.981210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ea2895e6-b3fb-43f5-bbde-bdacc175e304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:35.141298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2895e6-b3fb-43f5-bbde-bdacc175e304 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:35.149939 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:35.310290 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:35.311254 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:35.311636 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:35.312012 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:35.313809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:35.423082 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:35.424843 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b50de-915f-4d96-b81f-3129dead0b5f service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:35.445836 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 684/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:37.663241 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c4e2e-3cc4-4f53-959a-3e1fca0cb98d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:37.663796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c4e2e-3cc4-4f53-959a-3e1fca0cb98d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:37.663796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c4e2e-3cc4-4f53-959a-3e1fca0cb98d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:37.665324 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 696/1380] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:37.670577 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a12fe-d5ec-47b7-bec9-4dfd741693dd None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:37.671002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a12fe-d5ec-47b7-bec9-4dfd741693dd None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:37.671193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a12fe-d5ec-47b7-bec9-4dfd741693dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:37.682815 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:37.751521 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-434a12fe-d5ec-47b7-bec9-4dfd741693dd None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:37.752933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-434a12fe-d5ec-47b7-bec9-4dfd741693dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:37.864169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 685/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:37.909540 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c4e2e-3cc4-4f53-959a-3e1fca0cb98d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:37.911152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ce3c4e2e-3cc4-4f53-959a-3e1fca0cb98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:38.063141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:38.077190 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c4e2e-3cc4-4f53-959a-3e1fca0cb98d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:38.213657 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:38.214214 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:38.214588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:38.220993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:38.220993 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:38.319524 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:38.321101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf750d9a-9c21-463e-ade8-c6d3a3ccc429 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:38.332714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:38.341481 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 697/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:40.998655 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c28c9d-7757-48b3-858d-8d3e825d891f None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:40.999200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c28c9d-7757-48b3-858d-8d3e825d891f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:40.999200 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c28c9d-7757-48b3-858d-8d3e825d891f None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:41.000587 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 686/1383] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:41.006551 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69dcce-2330-45f8-85b8-a73e2b2de31d None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:41.007108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69dcce-2330-45f8-85b8-a73e2b2de31d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:41.007210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69dcce-2330-45f8-85b8-a73e2b2de31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:41.099646 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-9d69dcce-2330-45f8-85b8-a73e2b2de31d None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:41.101431 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9d69dcce-2330-45f8-85b8-a73e2b2de31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:41.225552 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 698/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:41.277039 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c28c9d-7757-48b3-858d-8d3e825d891f None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:41.279362 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c8c28c9d-7757-48b3-858d-8d3e825d891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:41.376828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:41.464801 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c28c9d-7757-48b3-858d-8d3e825d891f None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:41.590406 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:41.591062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:41.591344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:41.591616 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:41.593281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:41.648378 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:41.693777 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:41.695448 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d3234-cc1f-40f1-a45a-4e7b5eb1dc6c service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:41.715005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 687/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:43.950405 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344790c-7f63-4ab1-b0db-2406829d6430 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:43.951137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344790c-7f63-4ab1-b0db-2406829d6430 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:43.953499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344790c-7f63-4ab1-b0db-2406829d6430 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:43.955067 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 699/1386] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:43.960920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b812d76-0a6d-4872-944b-d6c23cf5f610 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:43.965472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b812d76-0a6d-4872-944b-d6c23cf5f610 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:43.965472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b812d76-0a6d-4872-944b-d6c23cf5f610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:44.055467 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-5b812d76-0a6d-4872-944b-d6c23cf5f610 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:44.057251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-5b812d76-0a6d-4872-944b-d6c23cf5f610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:44.205784 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 688/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:44.270496 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344790c-7f63-4ab1-b0db-2406829d6430 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:44.272895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-4344790c-7f63-4ab1-b0db-2406829d6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:44.399613 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:44.517164 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344790c-7f63-4ab1-b0db-2406829d6430 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:44.640479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:44.641125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:44.641408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:44.641682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:44.643923 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:44.684459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:44.742034 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:44.744880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9ee3-7164-40f3-b172-b5eb4019c691 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:44.769965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 700/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:47.039504 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c558be-f0c8-449e-9792-6beb2ac60d97 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.040074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c558be-f0c8-449e-9792-6beb2ac60d97 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.040074 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c558be-f0c8-449e-9792-6beb2ac60d97 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.041482 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 689/1389] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:47.046849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.047306 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.047415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.127797 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:47.129020 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:47.213066 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2992074-fe2e-42a7-99ad-f82517894f4d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.213575 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2992074-fe2e-42a7-99ad-f82517894f4d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.214080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2992074-fe2e-42a7-99ad-f82517894f4d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.216706 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 690/1390] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:55:47.222210 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128d52-a45e-4afb-846f-1d22fe7e60af None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.222502 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128d52-a45e-4afb-846f-1d22fe7e60af None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.222786 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128d52-a45e-4afb-846f-1d22fe7e60af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.233625 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 701/1391] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:47.269798 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:47.284050 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:47.285764 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:47.304491 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:47.312193 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-e5128d52-a45e-4afb-846f-1d22fe7e60af None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:47.316634 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-e5128d52-a45e-4afb-846f-1d22fe7e60af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:47.427006 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00438ad5-bbc4-445f-b7a8-33e84babae7c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:47.439712 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 691/1392] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:47.503277 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4233-5cc7-45f4-9009-72703a0c1087 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.508113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4233-5cc7-45f4-9009-72703a0c1087 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.508113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4233-5cc7-45f4-9009-72703a0c1087 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.508113 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 692/1393] 10.222.0.57 () {54 vars in 966 bytes} [Mon Jul 25 18:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:55:47.516728 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.517101 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.517460 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.579438 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:47.579985 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.580265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.580536 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.582447 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:47.593638 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:47.618417 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] MFA Rules not processed for user `86438daf324a458aa86b9051722c4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:47.621062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:47.677274 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:47.692668 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:47.694517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fc7f1-8d48-4aa1-93cc-c742830309f8 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:47.720724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 693/1395] 10.222.0.57 () {58 vars in 1063 bytes} [Mon Jul 25 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:47.728795 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 702/1395] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:47.731834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:47.739876 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd622d5-16ee-441c-a9ac-fec491c05555 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:47.826678 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86438daf324a458aa86b9051722c4d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:47.827344 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:47.827633 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:47.827906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:47.829845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:47.959320 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:47.960729 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0aa32-b7d1-43c9-9793-164d13193366 service glance] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:47.976308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:47.981108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 694/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:49.039459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71620e6-aea9-4dfe-9be2-6ca91cc510cf None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:49.040396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71620e6-aea9-4dfe-9be2-6ca91cc510cf None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:49.040559 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71620e6-aea9-4dfe-9be2-6ca91cc510cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:49.111918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-b71620e6-aea9-4dfe-9be2-6ca91cc510cf None None] MFA Rules not processed for user `78f5fc7f3ad048f08a78586de85af78e`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:49.113192 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-b71620e6-aea9-4dfe-9be2-6ca91cc510cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:49.207035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:49.210529 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 703/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 25 18:55:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:49.265245 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d6a37-9285-40cb-9011-ca38943ca6fb None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:49.265767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d6a37-9285-40cb-9011-ca38943ca6fb None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:49.265888 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d6a37-9285-40cb-9011-ca38943ca6fb None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:49.267281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 695/1398] 10.222.0.57 () {54 vars in 943 bytes} [Mon Jul 25 18:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:55:49.274742 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94512-c97e-4e3d-8605-7aea15117b65 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:49.275782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94512-c97e-4e3d-8605-7aea15117b65 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:49.275782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94512-c97e-4e3d-8605-7aea15117b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:49.372138 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-5df94512-c97e-4e3d-8605-7aea15117b65 None None] MFA Rules not processed for user `67c5071ac83e4e098a84afe77ce798ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:49.373891 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-5df94512-c97e-4e3d-8605-7aea15117b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:49.503416 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 704/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Mon Jul 25 18:55:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:49.513498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d6a37-9285-40cb-9011-ca38943ca6fb None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:49.515142 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a93d6a37-9285-40cb-9011-ca38943ca6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:49.680748 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d6a37-9285-40cb-9011-ca38943ca6fb None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:49.784158 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:49.819684 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67c5071ac83e4e098a84afe77ce798ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:49.820262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:49.820716 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:49.821001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:49.824203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:49.923929 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:49.925773 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6ce68-d037-4eb4-9628-aef783e89167 service swift] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:49.940228 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 696/1400] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 25 18:55:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 18:55:50.092187 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88fb10-70b1-4e52-92fc-d9dd2722dd14 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:50.092708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88fb10-70b1-4e52-92fc-d9dd2722dd14 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:50.092708 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88fb10-70b1-4e52-92fc-d9dd2722dd14 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:50.094163 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 705/1401] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:50.101667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ef0a-29c8-4f46-86a3-b194401d624f None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:50.101667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ef0a-29c8-4f46-86a3-b194401d624f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:50.101667 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ef0a-29c8-4f46-86a3-b194401d624f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:50.168638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-fd02ef0a-29c8-4f46-86a3-b194401d624f None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:50.169905 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-fd02ef0a-29c8-4f46-86a3-b194401d624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:50.240483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:50.279342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 697/1402] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:50.325982 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88fb10-70b1-4e52-92fc-d9dd2722dd14 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:50.328135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ae88fb10-70b1-4e52-92fc-d9dd2722dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:50.403135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:50.505631 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88fb10-70b1-4e52-92fc-d9dd2722dd14 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:50.648164 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:50.648769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:50.649057 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:50.649334 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:50.651081 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:50.683237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:50.799147 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:50.800961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22c413-ffac-42b7-8360-a6ad665f558c service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:50.827255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 706/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:53.272980 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65cf4e-7083-4f55-bfba-5c53ec17e3a2 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:53.273628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65cf4e-7083-4f55-bfba-5c53ec17e3a2 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:53.273722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65cf4e-7083-4f55-bfba-5c53ec17e3a2 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:53.275355 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 698/1404] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:53.282965 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fe147-df58-4c32-95c5-cb3e2c7c6720 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:53.283544 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fe147-df58-4c32-95c5-cb3e2c7c6720 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:53.283647 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fe147-df58-4c32-95c5-cb3e2c7c6720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:53.392881 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-570fe147-df58-4c32-95c5-cb3e2c7c6720 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:53.394316 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-570fe147-df58-4c32-95c5-cb3e2c7c6720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:53.504970 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:53.514571 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 707/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:53.573909 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65cf4e-7083-4f55-bfba-5c53ec17e3a2 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:53.600129 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1a65cf4e-7083-4f55-bfba-5c53ec17e3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:53.766617 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65cf4e-7083-4f55-bfba-5c53ec17e3a2 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:53.830281 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:53.894391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:53.895361 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:53.895642 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:53.895915 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:53.897604 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:53.997782 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:53.999501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120634e7-6a15-4d5d-9652-23604534e2c6 service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:54.021872 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 699/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:56.380626 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4e90e-ad64-42d5-b98f-d6e248fb2720 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:56.381196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4e90e-ad64-42d5-b98f-d6e248fb2720 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:56.381196 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4e90e-ad64-42d5-b98f-d6e248fb2720 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:56.386041 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 708/1407] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:56.388862 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f221-05d8-48c9-8a9c-845dd938fd0a None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:56.389970 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f221-05d8-48c9-8a9c-845dd938fd0a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:56.390500 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f221-05d8-48c9-8a9c-845dd938fd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:56.468794 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-6f55f221-05d8-48c9-8a9c-845dd938fd0a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:56.470062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6f55f221-05d8-48c9-8a9c-845dd938fd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:56.484313 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:56.601560 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 700/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:55:56.633410 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4e90e-ad64-42d5-b98f-d6e248fb2720 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:56.635516 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-45c4e90e-ad64-42d5-b98f-d6e248fb2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:56.803866 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4e90e-ad64-42d5-b98f-d6e248fb2720 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:56.888279 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:56.940116 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:56.940818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:56.941086 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:56.941362 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:56.943353 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:57.048910 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:57.050591 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fa127-08a5-4d41-8eff-ed095ba8112c service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:57.072495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 709/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:06.585551 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=38 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:07.069121 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=64 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:14.430704 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e2d3-8672-49b4-96c9-a5d3de46d0af None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:14.431813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e2d3-8672-49b4-96c9-a5d3de46d0af None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:14.431813 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e2d3-8672-49b4-96c9-a5d3de46d0af None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:14.433180 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 701/1410] 10.222.0.57 () {54 vars in 918 bytes} [Mon Jul 25 18:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:14.438499 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d7b14-abaf-4dae-a9ae-bb31a4ea8ee7 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:14.438975 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d7b14-abaf-4dae-a9ae-bb31a4ea8ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:14.439091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d7b14-abaf-4dae-a9ae-bb31a4ea8ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:14.516502 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-1c9d7b14-abaf-4dae-a9ae-bb31a4ea8ee7 None None] MFA Rules not processed for user `1e8c0d751eff42e9aa566ff36c19db84`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:14.517707 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1c9d7b14-abaf-4dae-a9ae-bb31a4ea8ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:14.627094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 710/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 25 18:56:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:14.641451 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22078c7a-609c-43f6-a1fc-08a59bfdaa6f None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:14.641740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22078c7a-609c-43f6-a1fc-08a59bfdaa6f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:14.642084 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22078c7a-609c-43f6-a1fc-08a59bfdaa6f None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:14.643485 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 702/1412] 10.222.0.57 () {54 vars in 918 bytes} [Mon Jul 25 18:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:14.648236 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bd089-664f-4724-86ab-3953bf9584c9 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:14.648475 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bd089-664f-4724-86ab-3953bf9584c9 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:14.648757 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bd089-664f-4724-86ab-3953bf9584c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:14.725216 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:14.727954 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-8c8bd089-664f-4724-86ab-3953bf9584c9 None None] MFA Rules not processed for user `2135d214a8274e60ac795f71a0697433`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:14.729223 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-8c8bd089-664f-4724-86ab-3953bf9584c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:14.847738 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 711/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 25 18:56:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:17.395246 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b502707-9e44-499d-81d8-a3e043479697 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:17.395668 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b502707-9e44-499d-81d8-a3e043479697 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:17.395901 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b502707-9e44-499d-81d8-a3e043479697 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:17.397153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:17.402800 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826d9e2-bea5-4cf3-b034-4f6297892977 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:17.403240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826d9e2-bea5-4cf3-b034-4f6297892977 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:17.403455 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826d9e2-bea5-4cf3-b034-4f6297892977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:17.472184 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-f826d9e2-bea5-4cf3-b034-4f6297892977 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:17.473420 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-f826d9e2-bea5-4cf3-b034-4f6297892977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:17.530237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:17.574855 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:17.617724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b502707-9e44-499d-81d8-a3e043479697 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:17.619904 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-5b502707-9e44-499d-81d8-a3e043479697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:17.775653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b502707-9e44-499d-81d8-a3e043479697 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:17.851746 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:17.892167 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:17.892814 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:17.893089 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:17.893358 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:17.895044 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:17.983977 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:17.985566 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e169490-c87a-4493-a5ff-532950d9ec1d service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:18.003807 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 704/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:19.024413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acaba6-663a-4cc1-9d2e-1b9ab4d50483 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:19.025644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acaba6-663a-4cc1-9d2e-1b9ab4d50483 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:19.025644 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acaba6-663a-4cc1-9d2e-1b9ab4d50483 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:19.026408 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:19.035312 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d331f6-f13c-4685-a2c0-4ad7d99189a1 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:19.035823 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d331f6-f13c-4685-a2c0-4ad7d99189a1 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:19.035992 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d331f6-f13c-4685-a2c0-4ad7d99189a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:19.127408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-14d331f6-f13c-4685-a2c0-4ad7d99189a1 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:19.128620 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-14d331f6-f13c-4685-a2c0-4ad7d99189a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:19.155672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:19.228959 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:19.378430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acaba6-663a-4cc1-9d2e-1b9ab4d50483 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:19.379874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-21acaba6-663a-4cc1-9d2e-1b9ab4d50483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:19.528049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acaba6-663a-4cc1-9d2e-1b9ab4d50483 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:19.569373 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:19.649867 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:19.651055 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:19.651332 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:19.651608 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:19.653314 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:19.755205 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:19.756794 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53857823-677e-4748-8c19-89e8eeaf54e0 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:19.776437 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 714/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:20.664736 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c22cc-a064-4e41-98bb-488ebcb11a64 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:20.665251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c22cc-a064-4e41-98bb-488ebcb11a64 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:20.665251 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c22cc-a064-4e41-98bb-488ebcb11a64 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:20.667171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:20.672288 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbdd09-d7b8-4c3c-9fba-f069d38a6c99 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:20.672674 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbdd09-d7b8-4c3c-9fba-f069d38a6c99 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:20.672856 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbdd09-d7b8-4c3c-9fba-f069d38a6c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:20.728731 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:20.749453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-35cbdd09-d7b8-4c3c-9fba-f069d38a6c99 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:20.750682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-35cbdd09-d7b8-4c3c-9fba-f069d38a6c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:20.848302 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:21.001974 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c22cc-a064-4e41-98bb-488ebcb11a64 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:21.003738 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-4c0c22cc-a064-4e41-98bb-488ebcb11a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:21.167501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c22cc-a064-4e41-98bb-488ebcb11a64 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:21.179640 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:21.280975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:21.281630 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:21.281911 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:21.282218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:21.283866 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:21.370423 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:21.372009 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b047b-c297-42fe-a3fc-8d82896467e9 service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:21.390063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 707/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:22.258430 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d413f4-0202-4596-ade3-6d05cec5542d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:22.259088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d413f4-0202-4596-ade3-6d05cec5542d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:22.259088 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d413f4-0202-4596-ade3-6d05cec5542d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:22.261137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:22.266556 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739c1f-b4ab-4ee6-b4e3-b7df3b27e847 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:22.267030 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739c1f-b4ab-4ee6-b4e3-b7df3b27e847 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:22.267145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739c1f-b4ab-4ee6-b4e3-b7df3b27e847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:22.287917 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:22.346829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-0e739c1f-b4ab-4ee6-b4e3-b7df3b27e847 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:22.348029 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-0e739c1f-b4ab-4ee6-b4e3-b7df3b27e847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:22.445975 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:22.601221 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d413f4-0202-4596-ade3-6d05cec5542d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:22.603482 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-96d413f4-0202-4596-ade3-6d05cec5542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:22.715271 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:22.752356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d413f4-0202-4596-ade3-6d05cec5542d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:22.866261 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:22.866827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:22.867108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:22.867379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:22.869694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:22.946510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:22.960167 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:22.961767 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf63514-1b8d-41ed-b1ab-95759775af9c service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:22.981374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 717/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:23.881155 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be734b-1c78-4980-9625-9877f56ef354 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:23.881724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be734b-1c78-4980-9625-9877f56ef354 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:23.881724 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be734b-1c78-4980-9625-9877f56ef354 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:23.883175 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:23.889135 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae3fff-185c-4657-abf6-c1523e1c73f2 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:23.889605 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae3fff-185c-4657-abf6-c1523e1c73f2 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:23.889714 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae3fff-185c-4657-abf6-c1523e1c73f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:23.971957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-d4ae3fff-185c-4657-abf6-c1523e1c73f2 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:23.973694 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-d4ae3fff-185c-4657-abf6-c1523e1c73f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:24.073233 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:24.228149 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be734b-1c78-4980-9625-9877f56ef354 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:24.229597 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-74be734b-1c78-4980-9625-9877f56ef354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:24.316322 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:24.380753 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be734b-1c78-4980-9625-9877f56ef354 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:24.494770 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:24.495341 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:24.495628 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:24.495902 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:24.497555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:24.546870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:24.588138 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:24.589795 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c12f0-490e-447e-bbe2-8784f1a63df3 service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:24.608621 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 710/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:25.491546 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c32e0-d9d5-463a-a509-0f91d2e1530a None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.492224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c32e0-d9d5-463a-a509-0f91d2e1530a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.492224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c32e0-d9d5-463a-a509-0f91d2e1530a None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.494107 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:25.499068 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a903-d82e-4269-830a-8df275b0c05f None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.499495 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a903-d82e-4269-830a-8df275b0c05f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.499626 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a903-d82e-4269-830a-8df275b0c05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.593800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-0b38a903-d82e-4269-830a-8df275b0c05f None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:25.595182 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-0b38a903-d82e-4269-830a-8df275b0c05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:25.699483 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:25.867473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c32e0-d9d5-463a-a509-0f91d2e1530a None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:25.870071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-109c32e0-d9d5-463a-a509-0f91d2e1530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:25.949747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:26.072346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c32e0-d9d5-463a-a509-0f91d2e1530a None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:26.224634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:26.225208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:26.225495 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:26.225774 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:26.228130 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:26.236724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:26.329453 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:26.331099 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3efa26-5d09-4580-a66d-ad5345353488 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:26.351852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 720/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:27.263785 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e263850-ca31-449f-a9f6-c91c1ffaaa58 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.264562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e263850-ca31-449f-a9f6-c91c1ffaaa58 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.264562 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e263850-ca31-449f-a9f6-c91c1ffaaa58 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.265744 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:27.271265 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5020da-242f-4235-b483-bc19270b630a None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.271777 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5020da-242f-4235-b483-bc19270b630a None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.271903 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5020da-242f-4235-b483-bc19270b630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.358246 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-ef5020da-242f-4235-b483-bc19270b630a None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:27.359488 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-ef5020da-242f-4235-b483-bc19270b630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.433183 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.466980 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:27.624325 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e263850-ca31-449f-a9f6-c91c1ffaaa58 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:27.625855 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6e263850-ca31-449f-a9f6-c91c1ffaaa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.644233 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.800208 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e263850-ca31-449f-a9f6-c91c1ffaaa58 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:27.898897 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.931508 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:27.932171 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.932574 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.932851 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.934559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:28.028950 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:28.030604 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b17bc-b605-4a24-a48f-24ed28c15395 service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:28.049128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 713/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:28.959456 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebafdc-1121-40f2-9d98-10a95991dbc2 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:28.960017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebafdc-1121-40f2-9d98-10a95991dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:28.960017 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebafdc-1121-40f2-9d98-10a95991dbc2 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:28.961379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:28.966972 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07af58-366b-4960-8e45-819239974d8d None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:28.970829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07af58-366b-4960-8e45-819239974d8d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:28.970829 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07af58-366b-4960-8e45-819239974d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:29.045684 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-8e07af58-366b-4960-8e45-819239974d8d None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:29.046962 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8e07af58-366b-4960-8e45-819239974d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:29.089948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:29.145723 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:29.299928 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebafdc-1121-40f2-9d98-10a95991dbc2 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:29.301450 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-3bebafdc-1121-40f2-9d98-10a95991dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:29.463679 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebafdc-1121-40f2-9d98-10a95991dbc2 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:29.519047 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:29.581972 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:29.582570 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:29.582849 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:29.583123 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:29.584788 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:29.699523 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:29.701125 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2830a-8030-44c5-a5c7-a5497d8ffb09 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:29.720149 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 723/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:30.601617 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ccf34-d315-4e20-8459-f769a4cf891d None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:30.602204 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ccf34-d315-4e20-8459-f769a4cf891d None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:30.602204 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ccf34-d315-4e20-8459-f769a4cf891d None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:30.603606 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:30.611917 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb384f07-4acb-4b6c-b046-5784228e22b1 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:30.612398 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb384f07-4acb-4b6c-b046-5784228e22b1 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:30.612497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb384f07-4acb-4b6c-b046-5784228e22b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:30.689622 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-bb384f07-4acb-4b6c-b046-5784228e22b1 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:30.690796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-bb384f07-4acb-4b6c-b046-5784228e22b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:30.692852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:30.790186 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:30.943672 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ccf34-d315-4e20-8459-f769a4cf891d None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:30.945169 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c90ccf34-d315-4e20-8459-f769a4cf891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:31.098203 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ccf34-d315-4e20-8459-f769a4cf891d None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:31.125650 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:31.214366 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:31.214932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:31.215213 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:31.215501 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:31.217141 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:31.326513 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:31.328153 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811be1a2-a255-4ef1-b88b-cb7a42719e54 service nova] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:31.348490 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 716/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:33.393449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e62a1-7861-4d7f-95e7-eeeaccd9d135 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:33.394251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e62a1-7861-4d7f-95e7-eeeaccd9d135 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:33.394251 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e62a1-7861-4d7f-95e7-eeeaccd9d135 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:33.395833 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:33.402740 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ddb0-56d3-4538-b71f-51de24d26c6e None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:33.403891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ddb0-56d3-4538-b71f-51de24d26c6e None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:33.403891 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ddb0-56d3-4538-b71f-51de24d26c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:33.466080 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:33.500619 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-9ee9ddb0-56d3-4538-b71f-51de24d26c6e None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:33.501917 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-9ee9ddb0-56d3-4538-b71f-51de24d26c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:33.657759 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:33.729541 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e62a1-7861-4d7f-95e7-eeeaccd9d135 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:33.731415 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-5c1e62a1-7861-4d7f-95e7-eeeaccd9d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:33.911748 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e62a1-7861-4d7f-95e7-eeeaccd9d135 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:33.960255 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:34.021287 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:34.022318 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:34.022737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:34.023185 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:34.025944 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:34.203203 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:34.205487 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8f9c3-718d-442d-b19c-abc90d368b7e service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:34.237027 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 726/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:37.029472 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c69d0-9df2-4392-82ec-0e9f86e8b684 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:37.030115 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c69d0-9df2-4392-82ec-0e9f86e8b684 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:37.030229 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c69d0-9df2-4392-82ec-0e9f86e8b684 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:37.031809 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:37.038376 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3ae4-7cdc-4dd8-b5ae-5f26c21b3667 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:37.039661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3ae4-7cdc-4dd8-b5ae-5f26c21b3667 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:37.039661 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3ae4-7cdc-4dd8-b5ae-5f26c21b3667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:37.131957 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-d13d3ae4-7cdc-4dd8-b5ae-5f26c21b3667 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:37.134237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-d13d3ae4-7cdc-4dd8-b5ae-5f26c21b3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:37.160801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:37.286043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:37.331679 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c69d0-9df2-4392-82ec-0e9f86e8b684 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:37.334839 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-6e4c69d0-9df2-4392-82ec-0e9f86e8b684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:37.484991 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c69d0-9df2-4392-82ec-0e9f86e8b684 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:37.501134 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:37.592737 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:37.594426 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:37.594722 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:37.595001 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:37.597332 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:37.730847 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:37.732306 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a4f29-8437-4017-8d0d-9fb618382a33 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:37.752054 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 719/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:40.374272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911754-f4c9-4fde-be10-1e5503066062 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:40.374852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911754-f4c9-4fde-be10-1e5503066062 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:40.374852 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911754-f4c9-4fde-be10-1e5503066062 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:40.376243 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:56:40.386493 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9488fd-c9d1-4afc-95b4-802fdad1ee78 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:40.387128 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9488fd-c9d1-4afc-95b4-802fdad1ee78 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:40.387365 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9488fd-c9d1-4afc-95b4-802fdad1ee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:40.479993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:40.502629 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-fb9488fd-c9d1-4afc-95b4-802fdad1ee78 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:40.503878 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-fb9488fd-c9d1-4afc-95b4-802fdad1ee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:40.629767 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:40.669580 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9911754-f4c9-4fde-be10-1e5503066062 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:40.671154 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a9911754-f4c9-4fde-be10-1e5503066062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:40.789865 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9911754-f4c9-4fde-be10-1e5503066062 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:40.840040 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:40.880213 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:40.880983 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:40.881517 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:40.882182 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:40.884036 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:41.008785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:41.010248 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fc2bd-34ec-407b-82fd-685379a797dc service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:41.030180 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 729/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:43.226654 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e056-f009-4c5d-9e12-51d3b65f9d49 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:43.227237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e056-f009-4c5d-9e12-51d3b65f9d49 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:43.227237 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e056-f009-4c5d-9e12-51d3b65f9d49 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:43.228624 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:43.233906 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebcc6c-f616-4bf8-9a3e-260dd5029e9f None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:43.234379 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebcc6c-f616-4bf8-9a3e-260dd5029e9f None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:43.234493 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebcc6c-f616-4bf8-9a3e-260dd5029e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:43.319911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-4eebcc6c-f616-4bf8-9a3e-260dd5029e9f None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:43.321765 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-4eebcc6c-f616-4bf8-9a3e-260dd5029e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:43.356309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:43.462514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:43.512376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e056-f009-4c5d-9e12-51d3b65f9d49 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:43.513928 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-2ac5e056-f009-4c5d-9e12-51d3b65f9d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:43.643216 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e056-f009-4c5d-9e12-51d3b65f9d49 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:43.677192 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:43.744669 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:43.745335 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:43.745643 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:43.745999 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:43.747763 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:43.892682 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:43.894089 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246d063-5d3e-4f3c-84ab-90dd3f97c092 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:43.916144 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 722/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:46.089388 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51648acf-db4e-40e8-9085-b52e2494649c None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:46.090035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51648acf-db4e-40e8-9085-b52e2494649c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:46.090035 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51648acf-db4e-40e8-9085-b52e2494649c None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:46.091356 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:46.097242 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa53d4-d301-46f2-8239-9881f5e19581 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:46.097793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa53d4-d301-46f2-8239-9881f5e19581 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:46.097793 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa53d4-d301-46f2-8239-9881f5e19581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:46.169889 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:46.173272 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-99fa53d4-d301-46f2-8239-9881f5e19581 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:46.175146 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-99fa53d4-d301-46f2-8239-9881f5e19581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:46.297262 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:46.338956 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51648acf-db4e-40e8-9085-b52e2494649c None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:46.340845 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-51648acf-db4e-40e8-9085-b52e2494649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:46.458685 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51648acf-db4e-40e8-9085-b52e2494649c None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:46.533947 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:46.581717 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:46.582991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:46.583824 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:46.584598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:46.588458 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:46.736724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:46.738108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae890c1-9056-481a-bb0a-5f9a38e77c8f service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:46.759533 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 732/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:49.117214 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724982-4c60-456a-8e20-f474b265b4da None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:49.117920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724982-4c60-456a-8e20-f474b265b4da None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:49.117920 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724982-4c60-456a-8e20-f474b265b4da None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:49.119194 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:49.125945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2605874-7713-4560-a1d8-4fa28f3ef7a7 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:49.126828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2605874-7713-4560-a1d8-4fa28f3ef7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:49.126828 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2605874-7713-4560-a1d8-4fa28f3ef7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:49.205443 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-c2605874-7713-4560-a1d8-4fa28f3ef7a7 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:49.206732 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c2605874-7713-4560-a1d8-4fa28f3ef7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:49.239537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:49.320445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:49.376480 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3724982-4c60-456a-8e20-f474b265b4da None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:49.378814 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-e3724982-4c60-456a-8e20-f474b265b4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:49.510447 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3724982-4c60-456a-8e20-f474b265b4da None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:49.549002 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:49.606638 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:49.607689 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:49.608152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:49.608530 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:49.611405 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:49.798736 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:49.800252 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936ae62-4888-4c42-853f-28ed2c1ed6a6 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:49.824196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 725/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:52.487926 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac4950-0f04-40bb-b109-e4f5f594e04d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:52.488537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac4950-0f04-40bb-b109-e4f5f594e04d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:52.488537 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac4950-0f04-40bb-b109-e4f5f594e04d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:52.489897 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:52.496151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4c63f-c756-4c21-a88e-20bb4a376c03 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:52.496749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4c63f-c756-4c21-a88e-20bb4a376c03 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:52.496749 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4c63f-c756-4c21-a88e-20bb4a376c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:52.581062 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-2dc4c63f-c756-4c21-a88e-20bb4a376c03 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:52.582337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-2dc4c63f-c756-4c21-a88e-20bb4a376c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:52.592842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:52.722656 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:52.763396 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac4950-0f04-40bb-b109-e4f5f594e04d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:52.765152 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a0ac4950-0f04-40bb-b109-e4f5f594e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:52.926688 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac4950-0f04-40bb-b109-e4f5f594e04d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:53.024412 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:53.058578 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:53.059706 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:53.060208 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:53.060696 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:53.064030 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:53.255490 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:53.257720 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101571d-d631-4ce5-92af-1a50f483ae02 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:53.281137 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 735/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:55.505671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e93e3-4ace-441d-b4ba-9b0f5a076182 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:55.506346 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e93e3-4ace-441d-b4ba-9b0f5a076182 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:55.506346 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e93e3-4ace-441d-b4ba-9b0f5a076182 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:55.507790 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:55.514401 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734be73c-f3d3-4490-9ed2-32386833594c None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:55.515023 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734be73c-f3d3-4490-9ed2-32386833594c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:55.515023 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734be73c-f3d3-4490-9ed2-32386833594c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:55.595104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-734be73c-f3d3-4490-9ed2-32386833594c None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:55.596474 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-734be73c-f3d3-4490-9ed2-32386833594c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:55.644369 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:55.742739 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:55.809647 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e93e3-4ace-441d-b4ba-9b0f5a076182 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:55.811243 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-919e93e3-4ace-441d-b4ba-9b0f5a076182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:55.939121 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e93e3-4ace-441d-b4ba-9b0f5a076182 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:56.003298 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:56.050560 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:56.051308 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:56.051731 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:56.052143 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:56.054482 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:56.222913 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:56.224552 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e5b63-9d9e-4844-a994-902686a3c9bf service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:56.246108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 728/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:56:58.499858 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6b45-ef08-4947-8d69-985c876b2284 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:58.500459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6b45-ef08-4947-8d69-985c876b2284 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:58.500459 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6b45-ef08-4947-8d69-985c876b2284 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:58.501810 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:58.507921 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627dc5-9a04-49af-8d69-7dcfd95e8f61 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:58.508492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627dc5-9a04-49af-8d69-7dcfd95e8f61 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:58.508646 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627dc5-9a04-49af-8d69-7dcfd95e8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:58.588005 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-33627dc5-9a04-49af-8d69-7dcfd95e8f61 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:58.589399 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-33627dc5-9a04-49af-8d69-7dcfd95e8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:58.612152 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:58.747133 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:58.800392 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6b45-ef08-4947-8d69-985c876b2284 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:58.803278 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-389d6b45-ef08-4947-8d69-985c876b2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:58.985911 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6b45-ef08-4947-8d69-985c876b2284 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:59.097237 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:59.107818 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:59.111998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:59.111998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:59.111998 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:59.113043 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:59.326329 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:59.328024 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31b807-051d-48e4-8ad4-fd201535e289 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:59.350355 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 738/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:01.666383 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88886c-6885-46e0-a077-596011a9d1b9 None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:01.667136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88886c-6885-46e0-a077-596011a9d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:01.667376 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88886c-6885-46e0-a077-596011a9d1b9 None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:01.669992 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:01.679634 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3fde3-a5f6-4ac1-8f83-56da94321423 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:01.680394 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3fde3-a5f6-4ac1-8f83-56da94321423 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:01.680918 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3fde3-a5f6-4ac1-8f83-56da94321423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:01.787782 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-81e3fde3-a5f6-4ac1-8f83-56da94321423 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:01.789091 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-81e3fde3-a5f6-4ac1-8f83-56da94321423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:01.853596 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:01.929585 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:01.972206 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88886c-6885-46e0-a077-596011a9d1b9 None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:01.974196 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1d88886c-6885-46e0-a077-596011a9d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:02.115403 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88886c-6885-46e0-a077-596011a9d1b9 None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:02.179350 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:02.213864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:02.215297 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:02.215834 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:02.216371 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:02.220554 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:02.352268 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:02.353887 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d22008-156e-4555-9e71-69cec0860550 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:02.375286 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 731/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:11.923532 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=22 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:12.372569 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=66 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:26.130589 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bdba-6af4-4ec6-8a1d-44d85b76819c None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.131471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bdba-6af4-4ec6-8a1d-44d85b76819c None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.131471 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bdba-6af4-4ec6-8a1d-44d85b76819c None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.132535 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 740/1471] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:26.139594 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a480c2-f3c4-4138-ad16-ae1a34a3e164 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.140178 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a480c2-f3c4-4138-ad16-ae1a34a3e164 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.140596 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a480c2-f3c4-4138-ad16-ae1a34a3e164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.250559 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-92a480c2-f3c4-4138-ad16-ae1a34a3e164 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:26.252279 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-92a480c2-f3c4-4138-ad16-ae1a34a3e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:26.398658 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 732/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:26.496907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c217a2-98cb-4218-a81c-da391302aafd None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.497240 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c217a2-98cb-4218-a81c-da391302aafd None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.497581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c217a2-98cb-4218-a81c-da391302aafd None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.499338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 741/1473] 10.222.0.57 () {54 vars in 966 bytes} [Mon Jul 25 18:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:26.505103 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8100c-c402-4195-a6d5-92cea5371fdf None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.505393 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8100c-c402-4195-a6d5-92cea5371fdf None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.505674 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8100c-c402-4195-a6d5-92cea5371fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.607038 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ece8100c-c402-4195-a6d5-92cea5371fdf None None] MFA Rules not processed for user `86438daf324a458aa86b9051722c4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:26.608800 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ece8100c-c402-4195-a6d5-92cea5371fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:26.634812 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:26.710610 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 733/1474] 10.222.0.57 () {58 vars in 1063 bytes} [Mon Jul 25 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:57:26.717062 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c217a2-98cb-4218-a81c-da391302aafd None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:26.719880 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-96c217a2-98cb-4218-a81c-da391302aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:26.877116 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c217a2-98cb-4218-a81c-da391302aafd None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:26.984089 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86438daf324a458aa86b9051722c4d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:26.985108 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.985594 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.986111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.989319 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:27.084445 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:27.171216 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:27.173133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fd018-edd5-44b3-b0f3-49d3fc2f45c0 service glance] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:27.195583 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 742/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:29.606494 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92af8-8034-47a0-a35e-b9673158400c None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:29.607090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92af8-8034-47a0-a35e-b9673158400c None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:29.607090 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92af8-8034-47a0-a35e-b9673158400c None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:29.608492 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 734/1476] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:29.615287 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99abbda-af5e-4164-9068-7f6e00522adf None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:29.615892 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99abbda-af5e-4164-9068-7f6e00522adf None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:29.615892 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99abbda-af5e-4164-9068-7f6e00522adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:29.711713 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-a99abbda-af5e-4164-9068-7f6e00522adf None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:29.713026 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-a99abbda-af5e-4164-9068-7f6e00522adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:29.840581 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 743/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:29.866526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92af8-8034-47a0-a35e-b9673158400c None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:29.868850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-bbb92af8-8034-47a0-a35e-b9673158400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:30.033517 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92af8-8034-47a0-a35e-b9673158400c None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:30.112870 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:30.172917 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:30.173598 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:30.173880 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:30.174260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] PATH_INFO: `/v3/projects` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:30.175019 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:30.176961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING py.warnings [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:57:30.176961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: warnings.warn(msg) Jul 25 18:57:30.176961 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: Jul 25 18:57:30.178160 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f606bc2-4cdf-4438-8e72-26383837f7ea admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:30.202714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 735/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 25 18:57:29 2022] GET /identity/v3/projects => generated 2553 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:57:32.631896 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7aad2-aa52-48f2-905b-fc72bddb0e44 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:32.632801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7aad2-aa52-48f2-905b-fc72bddb0e44 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:32.632801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7aad2-aa52-48f2-905b-fc72bddb0e44 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:32.635416 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 744/1479] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:57:32.645948 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4db27e-63d1-470b-bab1-7fdd8aed7bf5 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:32.646906 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4db27e-63d1-470b-bab1-7fdd8aed7bf5 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:32.647063 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4db27e-63d1-470b-bab1-7fdd8aed7bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:32.787027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-7e4db27e-63d1-470b-bab1-7fdd8aed7bf5 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:32.788410 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-7e4db27e-63d1-470b-bab1-7fdd8aed7bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:33.156259 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 736/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:33.231946 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7aad2-aa52-48f2-905b-fc72bddb0e44 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:33.240016 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-06e7aad2-aa52-48f2-905b-fc72bddb0e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:33.250682 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:33.439382 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7aad2-aa52-48f2-905b-fc72bddb0e44 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:33.580733 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:33.656224 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2135d214a8274e60ac795f71a0697433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:33.657394 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:33.657904 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:33.658416 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:33.662912 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:33.792049 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:33.795256 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eec0b-9d04-4b52-a706-f07567056f48 service nova] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:33.831953 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 745/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:36.713338 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccc7da-0e9d-45e8-ad27-13e0a184632a None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:36.714004 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccc7da-0e9d-45e8-ad27-13e0a184632a None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:36.714004 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccc7da-0e9d-45e8-ad27-13e0a184632a None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:36.715583 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 737/1482] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:36.726316 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ffe7c-2e3c-48fc-9c26-588f3c14d117 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:36.727085 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ffe7c-2e3c-48fc-9c26-588f3c14d117 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:36.727315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ffe7c-2e3c-48fc-9c26-588f3c14d117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:36.837786 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-7c6ffe7c-2e3c-48fc-9c26-588f3c14d117 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:36.839947 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-7c6ffe7c-2e3c-48fc-9c26-588f3c14d117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:36.931961 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:37.019796 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 746/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:37.084319 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccc7da-0e9d-45e8-ad27-13e0a184632a None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:37.086230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-91ccc7da-0e9d-45e8-ad27-13e0a184632a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:37.118756 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:37.234867 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccc7da-0e9d-45e8-ad27-13e0a184632a None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:37.334358 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:37.334900 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:37.335218 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:37.335528 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:37.337215 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:37.471097 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:37.548086 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:37.550479 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2f08e-e590-4f49-9baa-f9e3a49757d5 service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:37.579555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 738/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:57:40.046414 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1070e77-ea5f-4ab4-8363-636f0dff5476 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:40.047312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1070e77-ea5f-4ab4-8363-636f0dff5476 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:40.047312 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1070e77-ea5f-4ab4-8363-636f0dff5476 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:40.049227 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 747/1485] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:57:40.056177 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f918b-491a-4454-9efa-d4be8fb92394 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:40.056752 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f918b-491a-4454-9efa-d4be8fb92394 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:40.056903 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f918b-491a-4454-9efa-d4be8fb92394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:40.168438 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-3a7f918b-491a-4454-9efa-d4be8fb92394 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:40.169828 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-3a7f918b-491a-4454-9efa-d4be8fb92394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:40.318526 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 739/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:40.376607 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1070e77-ea5f-4ab4-8363-636f0dff5476 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:40.378530 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-c1070e77-ea5f-4ab4-8363-636f0dff5476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:40.558955 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1070e77-ea5f-4ab4-8363-636f0dff5476 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:40.690273 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:40.693230 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:40.694347 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:40.694814 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:40.695269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:40.698399 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:40.879272 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:40.881111 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fa336-57c9-4763-a92c-1309bad5edcc service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:40.904542 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 748/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:57:43.196884 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43784a-0d44-4f86-9960-b2525c083a4f None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:43.197498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43784a-0d44-4f86-9960-b2525c083a4f None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:43.197498 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43784a-0d44-4f86-9960-b2525c083a4f None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:43.198861 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 740/1488] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:43.204598 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9bbf7-e07f-4bf8-8909-5d528ea06d55 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:43.205101 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9bbf7-e07f-4bf8-8909-5d528ea06d55 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:43.205226 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9bbf7-e07f-4bf8-8909-5d528ea06d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:43.285141 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-dee9bbf7-e07f-4bf8-8909-5d528ea06d55 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:43.286417 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-dee9bbf7-e07f-4bf8-8909-5d528ea06d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:43.357895 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:43.408425 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 749/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:43.450973 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43784a-0d44-4f86-9960-b2525c083a4f None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:43.452457 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-1d43784a-0d44-4f86-9960-b2525c083a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:43.498303 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:43.602519 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43784a-0d44-4f86-9960-b2525c083a4f None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:43.712027 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:43.712714 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:43.712899 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:43.713168 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:43.715053 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:43.842145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:43.889211 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:43.890738 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac9fb6-3b51-4427-8ad6-a64cc8fe9bed service neutron] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:43.919595 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 741/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:46.613259 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc60e-63ec-4c00-9cb2-b931fee5ba8d None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:46.613945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc60e-63ec-4c00-9cb2-b931fee5ba8d None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:46.613945 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cc60e-63ec-4c00-9cb2-b931fee5ba8d None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:46.615297 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 750/1491] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 25 18:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:46.621120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba196ae2-287e-451e-b86c-0cf8020fda35 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:46.621886 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba196ae2-287e-451e-b86c-0cf8020fda35 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:46.622179 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba196ae2-287e-451e-b86c-0cf8020fda35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:46.723997 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-ba196ae2-287e-451e-b86c-0cf8020fda35 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:46.726145 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-ba196ae2-287e-451e-b86c-0cf8020fda35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:46.902951 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 742/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:46.977874 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cc60e-63ec-4c00-9cb2-b931fee5ba8d None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:46.979307 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-1a0cc60e-63ec-4c00-9cb2-b931fee5ba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:47.167724 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cc60e-63ec-4c00-9cb2-b931fee5ba8d None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:47.291719 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:47.292378 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:47.292681 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:47.292934 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:47.294801 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:47.307716 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=100 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:47.463281 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:47.465374 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef36c6-9023-42ec-bc33-10091f1a7c20 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:47.499315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 751/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:56.898774 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=82 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:57.494210 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=50 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:59:39.911270 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:39.911864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:39.911864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.001247 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:59:40.002561 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:40.122431 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 743/1494] 10.222.0.57 () {58 vars in 958 bytes} [Mon Jul 25 18:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:59:40.130879 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.131449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.131449 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.155510 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.auth.core [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:59:40.156769 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:40.165518 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 752/1495] 10.222.0.57 () {58 vars in 958 bytes} [Mon Jul 25 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:59:40.171083 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:40.180916 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e3e00-1ab7-4749-8580-8c207133f2ec None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:40.185845 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:40.186541 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.186821 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.187093 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.187856 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:59:40.188157 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:40.189184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING py.warnings [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:40.189184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: warnings.warn(msg) Jul 25 18:59:40.189184 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: Jul 25 18:59:40.190102 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:40.202060 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 744/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Jul 25 18:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:59:40.210479 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:40.221309 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862d446-15a3-48cd-b361-1063ff1cc8f4 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:40.226054 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:40.226718 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] REQUEST_METHOD: `POST` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.226991 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.227258 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.227950 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:40.229848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WARNING py.warnings [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:40.229848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: warnings.warn(msg) Jul 25 18:59:40.229848 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: Jul 25 18:59:40.231176 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:40.400407 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 753/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Jul 25 18:59:40 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:59:40.405571 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:40.416337 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32592c4c-046e-4ea0-853e-2d3ae48b18c5 admin admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:40.471288 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=100 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:59:40.478127 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:40.478864 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.479151 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.479430 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] PATH_INFO: `/v3/users` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.480130 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:40.481230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING py.warnings [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:40.481230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: warnings.warn(msg) Jul 25 18:59:40.481230 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: Jul 25 18:59:40.482432 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:40.596677 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 745/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 25 18:59:40 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:59:40.601602 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:40.613966 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd5e5-99d9-4476-aa4a-5978129f9e9c admin admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:40.619198 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:40.619884 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.620170 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.620451 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.621191 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:40.622388 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WARNING py.warnings [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:40.622388 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: warnings.warn(msg) Jul 25 18:59:40.622388 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: Jul 25 18:59:40.623409 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:40.637395 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 754/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Jul 25 18:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:59:40.642108 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:40.653555 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfdb36-6519-49e4-8844-4d9334a51014 admin admin] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:40.659703 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:40.662450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.662450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.662450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] PATH_INFO: `/v3/roles` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.662450 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:40.662819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING py.warnings [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:40.662819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: warnings.warn(msg) Jul 25 18:59:40.662819 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: Jul 25 18:59:40.663632 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f7d4e-1d70-4cc1-900e-3bc10ce1d321 admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:40.679015 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 746/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Jul 25 18:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:59:40.683269 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:40.694333 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09589-9d8d-4de2-a774-b896612d9c6b admin admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:40.699202 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:40.700104 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] REQUEST_METHOD: `PUT` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.700390 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.700737 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] PATH_INFO: `/v3/projects/72e0f44cf2794e3eae5390d089413a77/users/d90352fc43e74edfaa469192a307591a/roles/e587a7d6bedf4b6dabf88e847607ef97` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.718082 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e0f44cf2794e3eae5390d089413a77, user_id=d90352fc43e74edfaa469192a307591a, role_id=e587a7d6bedf4b6dabf88e847607ef97)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:40.719133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WARNING py.warnings [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:40.719133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: warnings.warn(msg) Jul 25 18:59:40.719133 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: Jul 25 18:59:40.721094 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec09e-9169-4a8f-810f-662d3a6ece6f admin admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:40.791574 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 755/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Jul 25 18:59:40 2022] PUT /identity/v3/projects/72e0f44cf2794e3eae5390d089413a77/users/d90352fc43e74edfaa469192a307591a/roles/e587a7d6bedf4b6dabf88e847607ef97 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:59:40.804342 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765608d9-53d9-48b1-bf22-8ee7f134f662 None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.804653 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765608d9-53d9-48b1-bf22-8ee7f134f662 None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.804933 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765608d9-53d9-48b1-bf22-8ee7f134f662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.836739 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-765608d9-53d9-48b1-bf22-8ee7f134f662 None None] MFA Rules not processed for user `d90352fc43e74edfaa469192a307591a`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:59:40.838189 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-765608d9-53d9-48b1-bf22-8ee7f134f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:40.937476 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 747/1502] 10.222.0.57 () {58 vars in 958 bytes} [Mon Jul 25 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:59:40.951747 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f124-741c-4bd1-a711-dadb17056644 None None] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.952234 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f124-741c-4bd1-a711-dadb17056644 None None] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.952473 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f124-741c-4bd1-a711-dadb17056644 None None] PATH_INFO: `/` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:40.953962 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 756/1503] 10.222.0.57 () {54 vars in 964 bytes} [Mon Jul 25 18:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:59:40.960327 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783024d-7ab0-4e61-a9f3-ef1a46d263fb None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:40.960671 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783024d-7ab0-4e61-a9f3-ef1a46d263fb None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:40.960993 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783024d-7ab0-4e61-a9f3-ef1a46d263fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:41.042551 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-8783024d-7ab0-4e61-a9f3-ef1a46d263fb None None] MFA Rules not processed for user `aa8107f1366e487f9e2ffc84b1ffb138`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:59:41.045191 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-8783024d-7ab0-4e61-a9f3-ef1a46d263fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:41.142413 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 748/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Jul 25 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:59:41.150497 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f124-741c-4bd1-a711-dadb17056644 None None] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:41.161935 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f124-741c-4bd1-a711-dadb17056644 None None] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:41.165619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa8107f1366e487f9e2ffc84b1ffb138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:41.166315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:41.166588 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:41.166861 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:41.177413 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:41.178971 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:41.190286 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 757/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:59:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:59:41.236836 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31865ea-cedf-4683-a4e6-dce17e8954ef None None] REQUEST_METHOD: `GET` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:41.237120 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31865ea-cedf-4683-a4e6-dce17e8954ef None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:41.237391 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31865ea-cedf-4683-a4e6-dce17e8954ef None None] PATH_INFO: `/` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:41.238733 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 749/1506] 10.222.0.57 () {52 vars in 830 bytes} [Mon Jul 25 18:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:59:41.242775 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:41.308582 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64c9d3-ba59-40b1-828b-b275fc97a07c service cinder] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:41.313863 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:41.314785 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:41.315071 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:41.315346 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] PATH_INFO: `/v3/users/d90352fc43e74edfaa469192a307591a` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:41.316567 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90352fc43e74edfaa469192a307591a)` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:41.317584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: WARNING py.warnings [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:41.317584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: warnings.warn(msg) Jul 25 18:59:41.317584 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: Jul 25 18:59:41.318827 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:41.568338 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90352fc43e74edfaa469192a307591a'} {{(pid=86941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:41.587326 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90352fc43e74edfaa469192a307591a'} {{(pid=86941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:41.663907 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90352fc43e74edfaa469192a307591a'} {{(pid=86941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:41.664539 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90352fc43e74edfaa469192a307591a'} {{(pid=86941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:41.674808 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:59:41.679301 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.notifications [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90352fc43e74edfaa469192a307591a'} {{(pid=86941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:41.685619 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 758/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Jul 25 18:59:41 2022] DELETE /identity/v3/users/d90352fc43e74edfaa469192a307591a => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:59:41.690260 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] REQUEST_METHOD: `POST` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:41.690550 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:41.690826 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:41.711408 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.auth.core [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] MFA Rules not processed for user `2ba410fb97cd412fa30f98b46a43dd28`. Rule list: `[]` (Enabled: `True`). {{(pid=86942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:59:41.712612 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:41.724231 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:59:41.776043 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 750/1508] 10.222.0.57 () {58 vars in 958 bytes} [Mon Jul 25 18:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:59:41.785648 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Authenticating user token {{(pid=86941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:41.787200 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.fernet_utils [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:41.868345 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9618cc-2141-45fa-8aaa-50aa911ee35a admin admin] Validating token access rules against request {{(pid=86941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:41.872022 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda25bd-40c5-4ee8-a544-dc5f47107519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c21d403c2d5498e99eaf13ae72d55ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47f153d7ef574e89b303a2d0b529fb40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:41.872745 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda25bd-40c5-4ee8-a544-dc5f47107519 service neutron] REQUEST_METHOD: `GET` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:41.873038 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda25bd-40c5-4ee8-a544-dc5f47107519 service neutron] SCRIPT_NAME: `/identity` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:41.873315 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda25bd-40c5-4ee8-a544-dc5f47107519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:41.882851 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda25bd-40c5-4ee8-a544-dc5f47107519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:41.884290 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda25bd-40c5-4ee8-a544-dc5f47107519 service neutron] RBAC: Authorization granted {{(pid=86941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:41.894514 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: [pid: 86941|app: 0|req: 759/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 25 18:59:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:59:42.385715 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] Authenticating user token {{(pid=86942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:59:42.388955 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.fernet_utils [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:59:42.401140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6cdeb-f840-4b4a-a708-04176a8454ba None None] Validating token access rules against request {{(pid=86942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:59:42.406462 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba410fb97cd412fa30f98b46a43dd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16218857a48d4501b2b8db4e0e5b32f7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:59:42.407404 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:59:42.407842 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:59:42.408186 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] PATH_INFO: `/v3/projects/72e0f44cf2794e3eae5390d089413a77` {{(pid=86942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:59:42.409471 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72e0f44cf2794e3eae5390d089413a77)` {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:59:42.410680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: WARNING py.warnings [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:59:42.410680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: warnings.warn(msg) Jul 25 18:59:42.410680 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: Jul 25 18:59:42.411863 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] RBAC: Authorization granted {{(pid=86942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:59:42.633136 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72e0f44cf2794e3eae5390d089413a77', 'request_id': 'req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8'} {{(pid=86942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:42.648963 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72e0f44cf2794e3eae5390d089413a77', 'request_id': 'req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8'} {{(pid=86942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:42.656331 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72e0f44cf2794e3eae5390d089413a77', 'request_id': 'req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8'} {{(pid=86942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:42.778850 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] The token cache is being invalidate because project 72e0f44cf2794e3eae5390d089413a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 18:59:42.779140 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:42.779932 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG keystone.notifications [None req-90c4e02c-3b7b-4c97-8915-eb71a570b9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 18:59:42.781506 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: [pid: 86942|app: 0|req: 751/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Jul 25 18:59:42 2022] DELETE /identity/v3/projects/72e0f44cf2794e3eae5390d089413a77 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:59:51.890339 dv-cinder-825492-29-iscsi devstack@keystone.service[86941]: DEBUG dbcounter [-] [86941] Writing DB stats keystone:SELECT=38 {{(pid=86941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:59:52.775049 dv-cinder-825492-29-iscsi devstack@keystone.service[86942]: DEBUG dbcounter [-] [86942] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}